-- Logs begin at Thu 2022-06-30 16:55:09 UTC, end at Thu 2022-06-30 18:46:06 UTC. -- Jun 30 17:38:35.141539 dv-cinder-812685-15-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 17:38:35.148852 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 17:38:35 2022] *** Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: nodename: dv-cinder-812685-15-iscsi Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: machine: x86_64 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: clock source: unix Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: pcre jit disabled Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: detected number of CPU cores: 4 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: current working directory: / Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: detected binary path: /usr/bin/uwsgi-core Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: your processes number limit is 47680 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: your memory page size is 4096 bytes Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: detected max file descriptor number: 2048 Jun 30 17:38:35.152493 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: lock engine: pthread robust mutexes Jun 30 17:38:35.153212 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: thunder lock: enabled Jun 30 17:38:35.153212 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 17:38:35.153212 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 17:38:35.197238 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: Python main interpreter initialized at 0x558e02c909c0 Jun 30 17:38:35.197238 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: python threads support enabled Jun 30 17:38:35.197238 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: your server socket listen backlog is limited to 100 connections Jun 30 17:38:35.197238 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: your mercy for graceful operations on workers is 90 seconds Jun 30 17:38:35.197571 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 17:38:35.197670 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: *** Operational MODE: preforking *** Jun 30 17:38:35.197946 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: *** uWSGI is running in multiple interpreter mode *** Jun 30 17:38:35.197946 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: spawned uWSGI master process (pid: 91843) Jun 30 17:38:35.198142 dv-cinder-812685-15-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 17:38:35.202485 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: spawned uWSGI worker 1 (pid: 91851, cores: 1) Jun 30 17:38:35.202485 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: spawned uWSGI worker 2 (pid: 91852, cores: 1) Jun 30 17:38:35.202485 dv-cinder-812685-15-iscsi devstack@keystone.service[91843]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 17:38:40.622346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 17:38:40.622346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: from cryptography.utils import int_from_bytes Jun 30 17:38:40.623155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 17:38:40.623155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: from cryptography.utils import int_from_bytes Jun 30 17:38:40.637451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 17:38:40.637451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: from cryptography.utils import int_from_bytes Jun 30 17:38:40.637910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 17:38:40.637910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: from cryptography.utils import int_from_bytes Jun 30 17:38:40.671761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 17:38:40.671936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 17:38:40.672124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] command line args: [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 17:38:40.672388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 17:38:40.672564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ================================================================================ {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 17:38:40.672815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] admin_token = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.673057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] config_dir = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.673241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.673494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] config_source = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.674218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] debug = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.674437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.674809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.674988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.675238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.675486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.675717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.675895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_config_append = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.676071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.676245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_dir = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.676422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_file = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.676750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.676901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.677147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.677387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.677565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.677732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.677973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.678216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.678479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.678711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.678940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.679169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.679412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.679655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.679926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.680402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.680580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] publish_errors = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.680805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.681026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.681217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.681444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.681691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.681926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] standard_threads = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.682135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.682403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.682579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.682752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] use_journal = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.682924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] use_json = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.683095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] use_stderr = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.683267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] use_syslog = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.683493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.683745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.683981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.684238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.684476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.684722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.684957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.685201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.685436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.685679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.685923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.686171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.686508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.687042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.687285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.687533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.687781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.688017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.688271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.688529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.688772 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.689020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.689438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.689643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.689897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.690147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.690387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 17:38:40.690517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.690693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 17:38:40.690817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.690940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] command line args: [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 17:38:40.691073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.691189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 17:38:40.691304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.691427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ================================================================================ {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 17:38:40.691543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.691663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] admin_token = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.691777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.691890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] config_dir = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.692010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.692175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.692292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.692412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] config_source = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.692412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] debug = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.692640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.692767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.692907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.693022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.693142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.693256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.693376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.693502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.693618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.693731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.693845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.693963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.694078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.694205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_config_append = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.694367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.694487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.694487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_dir = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.694714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.694827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_file = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.694952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.695067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.695180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.695293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.695414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.695536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.695657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.695777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.695894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.696007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.696128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.696250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.696366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.696479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.696593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.696707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.696821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.696945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.697061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.697180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.697297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.697411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.697531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.697728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.697850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.697971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.698091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.698211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.698325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.698486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.698604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.698604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.698818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.698941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.699056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] publish_errors = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.699424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.699539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.699652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.699778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.699778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.699991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.700105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.700230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.700345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.700459 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.700587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] standard_threads = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.700703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.700823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.700938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.701051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.701165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.701290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.701411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.701532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] use_journal = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.701532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] use_json = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.701754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.701868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] use_stderr = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.701992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.702107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] use_syslog = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.702227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.702380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 17:38:40.702496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.702610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.702805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.702923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.703957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.704929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.705982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.706861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.707976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.708925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.external = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.709960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.password = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.710892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] auth.token = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.711956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.712895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.713965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.714945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.715969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.716908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.717981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.718944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.719932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.720056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.720172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.720781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.720901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.721975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.722887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.723979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.724919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.725961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.726936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.727886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.external = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.password = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.728952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] auth.token = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.729988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.730991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.731925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.732955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.733787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.734899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.735837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.736560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.736685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.736802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.736928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.737994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.738953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.739906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.740974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.741921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.742908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.743941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] role.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] role.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.744979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.745899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.746979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.747915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.748983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.749919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.750896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.751979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.752981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.753929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.754969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.755910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.connection = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.756989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.757935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.758909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.759973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.760908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91852) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] role.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] role.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.761381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.762316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.763806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.764022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.764227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.764473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.764716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.764953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.765187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.765438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.765696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.765944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.766190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.766453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.766688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.766926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.767185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.767436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.767673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.767931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.768180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.768419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.768662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.768907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.769156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.769392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.769629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.769864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.770110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.770379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.770615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.770859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.771169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.771429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.connection = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.771676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.771924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.772167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.772411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.772648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.772884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.773118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.773354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.773598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.773835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.774102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.774314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.774591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.774830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.775075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.775307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.775550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.775787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 17:38:40.775959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91851) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 17:38:40.788856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.789092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.789316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.797635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.802402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.802638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.802864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.811363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.825939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.826170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.829466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.829795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.968341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:40.968598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.018658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.018896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.021017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.021255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.083742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.083953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.084890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.085017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.109901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.110963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.111047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.111181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.111264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.111396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.111609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.122219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.122452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.134892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.135980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.136112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.136194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91851) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.136323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.136532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91852) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 17:38:41.153855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.154046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.154132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 17:38:41.154254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 17:38:41.154411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.154612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 17:38:41.154695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.154807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 17:38:41.154890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.155833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.167961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.168041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.176139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.177268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.179393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.179669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.179948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.180225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.180500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.180595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.180752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.180835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.181881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.182060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.182143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.182363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.182625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.182895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.183159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.209514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.210403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.211131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.211407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.212007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.212286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.216585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.216860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.217249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.217505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.217782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.218173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.225088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.225323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.225973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.226253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.229737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.230054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.230360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.230640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.230735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.230912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.230993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.231784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.232041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.232307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.232570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.250026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.250302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.251049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.251325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.254939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.255842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.258544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.258835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.259522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.259819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.263889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.264915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.267989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.269043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.271589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.271882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.272167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.272445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.272687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.272782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.272986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.273069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.273269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.273549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.273843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.274123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.292315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.292849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.293118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.293377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.293624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.293903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.294168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.294524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.301170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.302379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.302872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.303147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.304065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.304343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.306305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.306602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.306878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.307155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.307425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.307542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.307830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.308098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.308377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.308653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.320510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.320784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.321059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.321335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.321620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.321877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.322152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.322459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.322740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.323029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.323155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.323458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.324812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.325107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.341796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.343432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.345028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.347251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.348940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.350556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.350672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.352280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.352363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.352559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.352833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.353897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.354175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.354482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.360180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.361784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.361870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.363484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.364972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.365266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.365654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.365929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.366719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.367016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.367412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.367698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.374979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.375261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.375536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.376762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.377047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.377326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.383670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91852) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.383953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.384228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.384500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.384766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91851) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 17:38:41.385410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.385950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.386870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91852) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.387955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.388216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.388479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.388744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91851) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 17:38:41.426218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558e02c909c0 pid: 91852 (default app) Jun 30 17:38:41.427313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558e02c909c0 pid: 91851 (default app) Jun 30 17:38:41.429548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94361-44f9-48fc-a3df-1353fc6270d7 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:41.429837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94361-44f9-48fc-a3df-1353fc6270d7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:41.430108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc94361-44f9-48fc-a3df-1353fc6270d7 None None] PATH_INFO: `/v3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:41.434267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Jun 30 17:38:41 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:49.038779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:49.039256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:49.039345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:49.043655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:38:49.049134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:49.049546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:49.049678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:49.128682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: INFO dbcounter [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] Registered counter for database keystone Jun 30 17:38:49.150532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG oslo_db.sqlalchemy.engines [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91852) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 17:38:49.156642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writer thread running {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 17:38:49.334147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG passlib.handlers.bcrypt [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91852) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 17:38:49.334594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG passlib.handlers.bcrypt [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91852) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 17:38:49.388544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:49.390157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:49.520858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 17:38:49.710146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:49.780148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: INFO dbcounter [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] Registered counter for database keystone Jun 30 17:38:49.800409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG oslo_db.sqlalchemy.engines [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91851) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 17:38:49.806567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writer thread running {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 17:38:49.929353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6f20-a3ab-4541-8925-c472f423cc94 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:49.933579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:49.934233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:49.934545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:49.934822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:49.942998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:49.994971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING keystone.common.rbac_enforcer.enforcer [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 17:38:49.995320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ef7e-6fa8-4156-8f55-a83f6f0640ad admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:50.005447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jun 30 17:38:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 17:38:50.011353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:50.026130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a06658-748c-4414-b0f4-1f567c24a83c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:50.030577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:50.031687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:50.031975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:50.032253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] PATH_INFO: `/v3/projects/2b7acc57e2754d7ab425d395278a2937` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:50.034431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] RBAC: Authorizing `identity:get_project(project_id=2b7acc57e2754d7ab425d395278a2937)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:50.087121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 17:38:50.087648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab854fb-96c7-4ee6-8cd8-50b607226fce admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:50.091938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Jun 30 17:38:50 2022] GET /identity/v3/projects/2b7acc57e2754d7ab425d395278a2937 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:52.094031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81906de8-2911-4eef-8355-3cdfe9bd8e6c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:52.094560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81906de8-2911-4eef-8355-3cdfe9bd8e6c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:52.094653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81906de8-2911-4eef-8355-3cdfe9bd8e6c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:52.098710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:52.104160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:52.104526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:52.104725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:52.123038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:52.124568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:52.132882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 17:38:52.155711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81906de8-2911-4eef-8355-3cdfe9bd8e6c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:52.167583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81906de8-2911-4eef-8355-3cdfe9bd8e6c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:52.171812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:52.172430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:52.172713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:52.172987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:52.180641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:52.182438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dc399-f0ce-477a-931b-7f25f1857690 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:52.191617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jun 30 17:38:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 17:38:52.196712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:52.206420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c4577-cd73-4aeb-99c0-40c6ba1d8f1d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:52.210744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:52.211587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:52.211872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:52.212151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:52.213524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:52.215381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790e787-99fe-4d28-b945-1e86c0d2a7a2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:52.216920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Jun 30 17:38:52 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:54.281891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cefb4-6c39-46b0-9183-5c9e9aed45c9 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.282393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cefb4-6c39-46b0-9183-5c9e9aed45c9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.282495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cefb4-6c39-46b0-9183-5c9e9aed45c9 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.283778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:54.288962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.289369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.289633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.305820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:54.307024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:54.328120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123a9e-b504-4b81-a1fa-11fb86698b64 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.328412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123a9e-b504-4b81-a1fa-11fb86698b64 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.328698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70123a9e-b504-4b81-a1fa-11fb86698b64 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.330704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:38:54.335641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.335940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.336223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.375047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG passlib.handlers.bcrypt [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91851) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 17:38:54.375462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG passlib.handlers.bcrypt [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91851) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 17:38:54.387164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:38:54.410251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.422119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:54.424015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:54.432950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:54.515356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300c3a-9e84-4528-820c-b3ffdc2ce475 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:54.525150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:38:54.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.549434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:54.550237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.550558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.550840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.562234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:38:54.574240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:38:54.575229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:38:54.576040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:38:54.581198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.648190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec5856-9d7a-4985-bd05-a48d564c3177 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:54.650492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:54.661284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:54.680842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf772aa3-a26f-4145-8b1f-de96090f6e29 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:54.687671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:54.689110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.689403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.689681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.714625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:54.715282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.715567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.715844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.716588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:38:54.716898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:54.718299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:54.729759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:54.735150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.762684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:54.764214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:54.837862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2158-96a6-4624-aaf1-6b9ca5894464 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:54.863842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:38:54.869349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.871477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:54.872678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:54.872783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:54.872783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:54.872926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:54.874380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:54.875858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:54.878091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:54.883357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:54.898191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:54.978461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1844da-097a-4c5d-95b9-b69800eaee1b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:54.988747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febf9c5-b798-4309-8e2d-216f22a8e438 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:55.020448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:55.021254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:55.021691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:55.021966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:55.027284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:55.028643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:55.028939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:55.029220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:55.030460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:55.031917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:55.038373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:55.039789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:55.044047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:55.045532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:38:55.066984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:38:55.067953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:38:55.068837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:38:55.068837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:38:55.068837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:38:55.068837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:38:55.068837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:38:55.074418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:55.122936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:55.136219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:55.143250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee146089-e6fd-43c4-b38b-bf9ed127a215 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:55.183505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca728dbf-33ff-42b9-ba1d-43ccd8f14118 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:55.184793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:55.185455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:55.185743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:55.186027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:55.186913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.domain.id=default, user.id=5c7b0226fd854776bbee2a2d0d0ee93a) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:38:55.187232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:55.188725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276268f-8f6a-40bf-8680-fca993a73b4b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:55.195697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jun 30 17:38:55 2022] GET /identity/v3/role_assignments?user.id=5c7b0226fd854776bbee2a2d0d0ee93a&scope.domain.id=default&role.id=1df4096f023c48ccab33e5939498b3da => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:55.222245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:55.223298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:55.223768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:55.224232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:55.224979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:38:55.225294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:55.226769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78e9e-05e8-458b-9831-68a008d3be9c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:55.248744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:57.320513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc42b03-23b3-428a-a408-0abf0250ec42 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.321532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc42b03-23b3-428a-a408-0abf0250ec42 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.322054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc42b03-23b3-428a-a408-0abf0250ec42 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.323675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:57.329350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.330162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.330688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.355356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:57.356932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:57.449578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:38:57.483862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc42b03-23b3-428a-a408-0abf0250ec42 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:57.629104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc42b03-23b3-428a-a408-0abf0250ec42 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:57.682068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:57.682923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.683208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.683482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.703203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-c40a2b56-a508-4b5d-9479-415c6322d174 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:38:57.711464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:38:57.712306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:38:57.713061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 13/24] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:38:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:38:57.720511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:57.793678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:57.863979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecadc7-08ae-4ae8-bf4a-6aec7ebb5ae7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:57.916469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:57.917067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.917351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.917621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.918390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:38:57.918700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:57.920086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:57.933864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf5c2-bbc5-4854-a021-e8e22b84cae3 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.934557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf5c2-bbc5-4854-a021-e8e22b84cae3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.935561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf5c2-bbc5-4854-a021-e8e22b84cae3 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.939323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 14/25] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:57.941415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 12/26] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:57.944070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e9f7f-0919-4545-b32d-fc3fb8b7fa7a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:57.944587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e9f7f-0919-4545-b32d-fc3fb8b7fa7a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:57.945087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e9f7f-0919-4545-b32d-fc3fb8b7fa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:57.946480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:57.960368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:57.975851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-356e9f7f-0919-4545-b32d-fc3fb8b7fa7a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:57.977083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-356e9f7f-0919-4545-b32d-fc3fb8b7fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:58.090470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbae2a3-7346-474f-adcf-24b0216ffb76 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:58.098100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 15/27] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:38:58.099743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee752e6-5c59-4f16-b60d-268f8c5f52f8 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.100280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee752e6-5c59-4f16-b60d-268f8c5f52f8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.100753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee752e6-5c59-4f16-b60d-268f8c5f52f8 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.102232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 16/28] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:58.104137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e321653-b577-4e16-b6ab-6a38d4579c8d None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.104626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e321653-b577-4e16-b6ab-6a38d4579c8d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.105097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e321653-b577-4e16-b6ab-6a38d4579c8d None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.105899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:58.107781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 17/29] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:58.113314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.114035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.114642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.134018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:58.135381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:58.149066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:58.150355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.150858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.151415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.153347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:58.155326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fda97-75c0-4269-a756-74a5b65891d9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:58.157387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 13/30] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:38:57 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:38:58.159673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.160229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.160713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.182816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:58.183950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:58.216103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 18/31] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:38:58.217389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.276507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 14/32] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:38:58.277695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.304590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:58.363154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905874e-3d23-4582-bfe8-67091d632f9a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:58.400340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cf0e-1b65-4e49-a959-4c150cd76839 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:58.405981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:58.407055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.407538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.408006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.409109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:58.410012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:58.411711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:58.444447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:58.445447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.445873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.446325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.447703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:58.449721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:58.451533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 15/33] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:38:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:58.452491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.561363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def3524-37ec-4391-89a6-6b9a16c66e1e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:58.564238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 19/34] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 17:38:58 2022] POST /identity/v3/roles => generated 225 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:38:58.565419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.640516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:58.642384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.643046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.645798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.675765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:58.677874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:38:58.714557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:38:58.715444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:38:58.716228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:38:58.716228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:38:58.716228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:38:58.716228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:38:58.726390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed5195-4863-4785-92a5-427a161093fd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:58.728242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 16/35] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:38:58.729310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.777299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:58.778941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:58.807612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:58.808419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:58.808713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:58.808994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:58.809759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:58.811316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:58.836024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 20/36] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 17:38:58 2022] POST /identity/v3/roles => generated 219 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:38:58.837215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:58.928157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77410-7ff2-4bfa-a34b-bd9914ec4afa None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.016972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.018039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.018451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.018739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] PATH_INFO: `/v3/domains/default/users/5c7b0226fd854776bbee2a2d0d0ee93a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.024925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5c7b0226fd854776bbee2a2d0d0ee93a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.027799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1df727-ff55-4a0e-ac1c-d0c11b76b21f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.040718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1f630-0c93-4fab-b474-37fc76ed88d9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.108096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.108847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.109140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.109426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.110250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:38:59.111274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.113156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.135950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 21/37] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:59.142182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:59.181988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 17/38] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Jun 30 17:38:58 2022] PUT /identity/v3/domains/default/users/5c7b0226fd854776bbee2a2d0d0ee93a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:38:59.189969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6752d-ca92-4285-a15f-5a9d9c6352e6 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.190660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6752d-ca92-4285-a15f-5a9d9c6352e6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.191150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6752d-ca92-4285-a15f-5a9d9c6352e6 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.196132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 18/39] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:38:59.208692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637f68d-a2d9-4c44-9174-2db72f27acf2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.209017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637f68d-a2d9-4c44-9174-2db72f27acf2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.209275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637f68d-a2d9-4c44-9174-2db72f27acf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.230916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d637f68d-a2d9-4c44-9174-2db72f27acf2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:59.232635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d637f68d-a2d9-4c44-9174-2db72f27acf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:59.312561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 19/40] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:38:59.313982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c921-a3bb-4154-bb3e-0e02e47b9e74 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.314278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c921-a3bb-4154-bb3e-0e02e47b9e74 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.314601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c921-a3bb-4154-bb3e-0e02e47b9e74 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.318373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 20/41] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:38:59.323142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:59.324364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.324654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.324991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.352590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:38:59.353810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:38:59.365578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:59.390549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf17f9-e2e9-4cca-903c-8d7a7dd6cd64 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.425882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.426569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.426867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.427144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.427837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.429286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.438959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 21/42] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:38:59.438959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:59.478368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 22/43] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:38:59 2022] POST /identity/v3/projects => generated 301 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:38:59.478368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:59.561841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c1822-a6f4-4913-8512-ecfcf4600aa6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.610868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.611762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.612124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.612502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.613903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.615665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.617106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:59.617577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 22/44] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:38:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:38:59.623480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:38:59.625442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c14ce7-a36e-4726-8622-7e68669c1dab None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.642957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:38:59.671103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.672541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.673055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.673484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.674787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.677024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0b37e-44bd-420f-b9e4-16b4c434731d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.700211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 17:38:59 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:38:59.755916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45ef31-accd-438c-b0b5-49e57f05eec4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:38:59.793377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:38:59.794053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:38:59.794354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:38:59.794638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:38:59.795363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:38:59.796841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa6ebc-5973-48fc-bf37-3f2a17ffeeb1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:38:59.828077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:38:59 2022] POST /identity/v3/projects => generated 312 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:01.248854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99121d-7580-472e-88a7-07fb38419afd None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.249339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99121d-7580-472e-88a7-07fb38419afd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.249339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99121d-7580-472e-88a7-07fb38419afd None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.251287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:01.256324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67532d-c727-4063-8861-55009946ddbd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.256723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67532d-c727-4063-8861-55009946ddbd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.256910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67532d-c727-4063-8861-55009946ddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.274064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7a67532d-c727-4063-8861-55009946ddbd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:01.275328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7a67532d-c727-4063-8861-55009946ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:01.334831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:01.343210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:01.366716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99121d-7580-472e-88a7-07fb38419afd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:01.466304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99121d-7580-472e-88a7-07fb38419afd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:01.498697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:01.499534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.499770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.500048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.510348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:01.517964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:01.518882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:01.519718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:01.524909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67532d-c727-4063-8861-55009946ddbd None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:01.622090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67532d-c727-4063-8861-55009946ddbd None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:01.656341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:01.657544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.657833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.658114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.658892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:01.659207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:01.660595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:01.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 25/50] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:01.675007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:01.693706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:01.773986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd75fa-04db-4f04-a4d6-308cac10270a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:01.806733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:01.807675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.807958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.808235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.809420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:01.810901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd159a2-0190-4e94-b2d3-14117eed311d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:01.812519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:01 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:01.817863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:01.858688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2de42-09ab-45a8-8e92-5759bd697022 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.858976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2de42-09ab-45a8-8e92-5759bd697022 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.859256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2de42-09ab-45a8-8e92-5759bd697022 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.861235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 27/52] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:01.866850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:01.867136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:01.867413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:01.885225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:01.886825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:01.955593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 28/53] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:01.979924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:01.988787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:01.995813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:02.030488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9ffa-f0e4-4f08-8223-407550e585b7 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:02.064510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:02.065295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:02.065580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:02.065856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:02.067001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:02.068447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:02.070059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 26/54] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:02.075860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:02.081152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b8f19-5f56-4d5d-97eb-f3d7a1ec50c9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:02.114394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:02.115192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:02.115480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:02.115761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:02.116939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:02.118424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:02.120053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 29/55] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:02.125334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:02.172325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ccca1-3dc8-42f4-8f2e-ae18f37bdf3d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:02.196587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:02.204571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:02.206458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:02.207113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:02.207395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:02.207672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:02.208498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.domain.id=default, user.id=5c7b0226fd854776bbee2a2d0d0ee93a) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:02.208808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:02.210226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc05fcc-6158-4dea-ae93-a7d72c306ad9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:02.218773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 27/56] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jun 30 17:39:02 2022] GET /identity/v3/role_assignments?user.id=5c7b0226fd854776bbee2a2d0d0ee93a&scope.domain.id=default&role.id=1df4096f023c48ccab33e5939498b3da => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:02.223279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd506568-2b04-48c2-89bd-b959a5e34d55 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:02.259051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:02.259731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:02.260006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:02.260286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:02.261006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:02.262538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f2029-83c0-4997-b848-bb5236356b28 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:02.300622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 30/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jun 30 17:39:02 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:39:04.323829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7410f-9c1d-452d-a33e-e6f2f524e061 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:04.324569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7410f-9c1d-452d-a33e-e6f2f524e061 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:04.324654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7410f-9c1d-452d-a33e-e6f2f524e061 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:04.326031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 28/58] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:04.331679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:04.332078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:04.332254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:04.349101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:04.350292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:04.413606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 31/59] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:04.437560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7410f-9c1d-452d-a33e-e6f2f524e061 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:04.538496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7410f-9c1d-452d-a33e-e6f2f524e061 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:04.573509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:04.574631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:04.574867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:04.575149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:04.576319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:04.577781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d704b77-f8af-4fd5-8b65-5806d5c5b442 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:04.579455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 29/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:04.584926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:04.643456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:04.682320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b58f1-4537-481a-822f-8eb052e3b465 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:04.717114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:04.717780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:04.718067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:04.718439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:04.719212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:04.721169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fdc4b-6fd4-400b-893a-adfde478aa03 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:04.746533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 32/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:04 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:39:07.674927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598836a5-2681-4f1a-b0c8-71f976d60e87 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:07.675915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598836a5-2681-4f1a-b0c8-71f976d60e87 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:07.677558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598836a5-2681-4f1a-b0c8-71f976d60e87 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:07.677970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 30/62] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:07.683465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175168d5-656e-4171-9a6f-df667378e37b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:07.683852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175168d5-656e-4171-9a6f-df667378e37b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:07.684028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175168d5-656e-4171-9a6f-df667378e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:07.711417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-175168d5-656e-4171-9a6f-df667378e37b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:07.712650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-175168d5-656e-4171-9a6f-df667378e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:07.808607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 33/63] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:07.843789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598836a5-2681-4f1a-b0c8-71f976d60e87 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:07.924556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:07.959594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcdd65-4805-44e6-8de6-c41f775ca476 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:07.960194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcdd65-4805-44e6-8de6-c41f775ca476 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:07.960667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcdd65-4805-44e6-8de6-c41f775ca476 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:07.962450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 34/64] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:07.967923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:07.968436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:07.968556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598836a5-2681-4f1a-b0c8-71f976d60e87 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:07.969029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.005444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.006495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.007073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.007486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.018803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:08.020018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:08.020591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:08.030800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:08.032386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:08.033290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:08.035274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 31/65] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:08.044505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.102796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.122414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 35/66] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:08.163442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.175097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b20dd3-52e5-4d94-ae3f-06da468c99b8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.211372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.212365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.213001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.213425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.214361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:08.218523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.220176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.221809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad981f-17e3-424c-9cbd-70d8ad38a4f4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.236489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 32/67] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:08.240607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083111e5-0500-46e6-800c-617cc2fae0b0 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.242652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083111e5-0500-46e6-800c-617cc2fae0b0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.244248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083111e5-0500-46e6-800c-617cc2fae0b0 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.245757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 33/68] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:08.246367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083111e5-0500-46e6-800c-617cc2fae0b0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.317891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422f769-2f38-4478-8413-6bcd6b775a7b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.361400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.362367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.362664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.362943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.375683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:08.381805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083111e5-0500-46e6-800c-617cc2fae0b0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8cc9924c-7a45-4a8d-b6dc-04c7b26a3b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:08.385373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:08.386227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:08.387028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 36/69] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:08.388106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.388106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.388399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.408994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:08.410196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:08.432352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.433962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.434513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.434994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.453173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.458647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.460486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.463814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 34/70] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:08.465383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.502370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 37/71] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:08.502370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.529276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.593394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff85b61-9df3-4938-858a-c2746c11103f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.614896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489cba4-8b9d-49b9-957f-28542045b086 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.626486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.627153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.627438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.627715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.628490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:08.628795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.630175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.638041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 35/72] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:08.638821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.648304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.648956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.649244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.649521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.687606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.717387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.718917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.738295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.741047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fed1ef-b92e-4622-abfe-62386108a69b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.774535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.775331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.775619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.775900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.785775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cd06a261-41fa-4902-9fd1-ac13cefc06c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:08.793718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:08.794571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:08.795340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 36/73] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:08.796108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c762-4de7-4094-b8cf-c0163f31d60b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.796396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c762-4de7-4094-b8cf-c0163f31d60b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.796676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4c762-4de7-4094-b8cf-c0163f31d60b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.798486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 37/74] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:08.800737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf811-3106-4796-bc44-ea071f9c481e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.801036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf811-3106-4796-bc44-ea071f9c481e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.801317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf811-3106-4796-bc44-ea071f9c481e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.802591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 38/75] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:08.803283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf811-3106-4796-bc44-ea071f9c481e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.821074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 38/76] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:08.822011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:08.889203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:08.913974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bf811-3106-4796-bc44-ea071f9c481e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.920429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679db5ec-e7b9-47f1-83db-03d78366a9be None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:08.947824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.948701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.949251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.949532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.950757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.952211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5442a-558d-4348-a52e-ede8c95a4dd6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.954043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 39/77] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:08.954167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:08.957075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.957184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.957254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.957318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.957402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.957470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.957534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:08.957534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:08.957717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c6eed-5c33-4058-b106-6883df1d6e60 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:08.966943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:08.968269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:08.968512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:08.968792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:08.976690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:08.977901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:08.984575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:08.986270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:09.003253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.045947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 40/79] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:09.046792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.055283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 40/80] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:09.056113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.130259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.153208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f023-bef8-4431-9dae-7c6f887972ac None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.173187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3912e2c-f9ad-49cd-b233-7de0a7509f69 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.191024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.191892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.192182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.192470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.193666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.195164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.196863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.197621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.207200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.207849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.208136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.208416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.225347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.274246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.275741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.296697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835a8b-0498-4b01-92f9-20030b71948f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.330613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.331475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.331767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.332057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.333126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.333229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.334682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.336567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.337326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.375123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 41/83] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:09.376458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.431114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.438734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4f268-00bf-4868-b496-908e585dbfdd None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.471382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0238f-8b84-4d8a-982b-82adde2e4c8b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.472074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.472856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.473137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.473414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.483361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:09.490968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:09.491870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:09.492637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 43/84] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:09.493270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.506030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.506851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.507144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.507418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.508591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.510045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.511694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 42/85] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.512450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.541233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.592842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf38cf-aac5-4578-b3da-e1413aa90654 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.610617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456ba1-6669-43dd-8db5-5a3cd662fd10 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.626050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.626734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.627016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.627288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.628239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:09.628553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.630002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.635557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 44/86] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.636334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.655672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.656436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.656792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.657144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.657620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.734365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.736354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.752281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839090d5-4580-457b-ba1b-22c1ad1f0849 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.768256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.799907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.800780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.801063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.801348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.802573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.804032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.810398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 45/87] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.810398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.855124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 43/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:09.855124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.900290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:09.916029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b90bd1-930f-45d6-a1f8-425bc787365d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.953344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:09.954069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:09.954372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:09.957205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:09.957902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:09.958217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:09.959683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:09.961282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f71bdd-d225-4df0-8549-0f0c88ac8ce7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:09.972926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 46/89] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:09.972926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:09.997536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.004189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.005177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.005569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.006174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.007672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.026737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 44/90] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:09 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:10.027836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:10.108866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e879f3-0a42-46f1-99b6-54e2a53733c0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.151797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.152692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.153108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.153495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.154498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5c7b0226fd854776bbee2a2d0d0ee93a) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:10.155415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.157066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.170451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:09 2022] GET /identity/v3/role_assignments?user.id=5c7b0226fd854776bbee2a2d0d0ee93a&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:10.170451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:10.190700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfee9-c813-47cc-ad6b-db3a3da06961 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.197581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.237314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.237919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.238203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.238559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.239929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.241594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.243536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 45/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:10.249363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:10.303230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef18735-914f-4954-bf45-a4a4def07364 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.315148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.345607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.346896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.347392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.347869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.349999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.351758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.354012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 48/93] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:10.361091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:10.372292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37272-2994-48a1-8c87-53fdcbce321f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.408981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.410917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.410917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.410917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.411380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:10.411700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.413473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.421824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 46/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:10.462158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.484136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba63ba-8383-4353-8f64-af5443fa786e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.524735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.525799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.526306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.526822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.604880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.606389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.691154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.711987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 49/95] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:10.717456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:10.874090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabd57b-a951-49ad-826d-ed46367c841d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:10.892418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:10.915769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:10.916657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:10.916939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:10.917213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] PATH_INFO: `/v3/projects/9af495954bc243e797f6e697f4005461` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:10.918514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] RBAC: Authorizing `identity:get_project(project_id=9af495954bc243e797f6e697f4005461)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:10.919982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524b726-7917-48a8-a8a0-24dd99317212 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:10.922002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 47/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/projects/9af495954bc243e797f6e697f4005461 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:10.928295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:11.049441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb023ed-63df-4b2a-90a9-569e8b98c871 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:11.089758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:11.090513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:11.090803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:11.091082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:11.091922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=9af495954bc243e797f6e697f4005461, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:11.092240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:11.093701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144546cf-41bf-4b61-af89-20a37d5292a6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:11.100440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 50/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:10 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=9af495954bc243e797f6e697f4005461&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:12.045721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb463-e2f6-4df0-8d27-06ec0fc74749 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.046218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb463-e2f6-4df0-8d27-06ec0fc74749 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.046291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deb463-e2f6-4df0-8d27-06ec0fc74749 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.048576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 48/98] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:12.053829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.054233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.054433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.071690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:12.072891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:12.129588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:12.164691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:12.204639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deb463-e2f6-4df0-8d27-06ec0fc74749 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:12.337486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deb463-e2f6-4df0-8d27-06ec0fc74749 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:12.371386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:12.372524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.373082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.373550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.384463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:12.393099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:12.393933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:12.394705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:12.397008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 49/100] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:12.403178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:12.518944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03d31d-c642-49c4-9a0f-5c7fa0a6125a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:12.557737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:12.558438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.558730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.559008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.559757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:12.560068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:12.561443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:12.570788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 52/101] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:12.576727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:12.665413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:12.731648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86a86-3e61-48c1-b381-1394a55f51b9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:12.769899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:12.770811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.771101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.771380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.772584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:12.774093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:12.775754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 50/102] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:12 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:12.781355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:12.841726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:12.924429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb033fde-e858-4795-b2a8-e0845e229855 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:12.964936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:12.964936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:12.964936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:12.964936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:12.965318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:12.965318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042edf4d-a2a4-4a8c-891c-9bfc3281ba3f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:12.965318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 53/103] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:12 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:12.970930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:13.057364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6a54c-c064-40e1-9503-b324d437f276 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.057862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6a54c-c064-40e1-9503-b324d437f276 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.057935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6a54c-c064-40e1-9503-b324d437f276 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.059334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 54/104] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:13.076014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.076311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.076600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.107458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:13.108935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:13.153359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d8b6-59e6-4568-b06f-51c4abec6958 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:13.182786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:13.253355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 55/105] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:13.267123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:13.268086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.268376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.268656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.272911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, user_id=5d213696814d4a9aaf5ee9a25ff03fa4, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:13.274564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:13.284147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:13.314447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 51/106] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:12 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:13.344885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:13.425592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be796eb-7115-4139-a8d7-297d98168de3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:13.461856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:13.462662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.462946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.463223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.479005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c265b8ca-ca07-4968-9884-cdae47485478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:13.487318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:13.488232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:13.488992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 56/107] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:13.495749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:13.619245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653949-760d-4f7c-a038-dad71f712b27 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.619566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653949-760d-4f7c-a038-dad71f712b27 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.619813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653949-760d-4f7c-a038-dad71f712b27 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.621508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 57/108] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:13.628097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.628531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.628818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.654132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:13.655354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:13.665100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8b56-062c-4850-a7ac-a4869ed52ed4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:13.722512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:13.723106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.723388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.723667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.726358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:13.726821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:13.730047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3662cd-87f4-4ca0-9e34-03e04e1ee333 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:13.749007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 52/109] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:13.751175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e957-a648-431f-9022-8fbe71eca29e None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.752051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e957-a648-431f-9022-8fbe71eca29e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.752340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94e957-a648-431f-9022-8fbe71eca29e None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.756798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 58/110] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:13.757532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:13.764735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 53/111] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:13.769941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.770228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.770532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.803385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:13.804624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:13.818725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:13.851896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:13.896937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 54/112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:13.903063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:13.915170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1600f6-d772-4171-8dfe-ea3c036c56f2 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:13.955843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:13.956563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.956955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.957304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.958620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:13.960156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432b734-9aa5-4c0f-aa6e-649054841a03 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:13.962085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 59/113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:13 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:13.967782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555fac-484d-4ddd-897e-60b943c40815 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:13.968185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555fac-484d-4ddd-897e-60b943c40815 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:13.968543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555fac-484d-4ddd-897e-60b943c40815 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:13.970459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 60/114] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:13.971577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04555fac-484d-4ddd-897e-60b943c40815 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.021277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8c509-1555-47ec-b1ef-0e62ca1392e9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.068547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.069596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.070056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.070510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.071843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.073441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.075533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 55/115] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:14.076531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.108820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.114105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.121461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04555fac-484d-4ddd-897e-60b943c40815 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.161039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.162191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.162705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.163152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.175305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-fcb6919d-b723-4926-800f-3712079215d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:14.184612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:14.185489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:14.186255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:14.186943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadd35e-36e1-4adb-ba83-5782d4ea4939 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.189253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 61/116] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:14.190719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.191327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.191802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.210381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:14.211939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:14.228375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.229357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.229638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.229921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.231152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.232708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.234760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 56/117] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:14.235348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.289793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 62/118] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:14.290965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.370842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224d5c-f3e1-4db2-90f3-14fc710c0ab0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.373564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.380117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.402145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2d38b-25b0-4e45-a3ea-0c3c75dcda01 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.413674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.414370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.414580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.414811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.415525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.417087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.443741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.443918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 57/119] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:39:14 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:14.444620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.444745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.445121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.445451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.446207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:14.446539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.449414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.466368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 63/120] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:14.467700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.576229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9204a0-3985-47d4-9f24-9dad1b8a4c8a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.599077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672853d1-b64a-4e94-bdf8-7120c0a4aa85 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.616276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.617218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.617510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.617788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5c7b0226fd854776bbee2a2d0d0ee93a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.621488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, user_id=5c7b0226fd854776bbee2a2d0d0ee93a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.623193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f17cd-64ec-43be-ba2c-ded132ed5836 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.638996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 58/121] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:14 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5c7b0226fd854776bbee2a2d0d0ee93a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:14.642399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6bc4-3333-4866-8ca3-e69d8e8e4959 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.642399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6bc4-3333-4866-8ca3-e69d8e8e4959 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.642399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b6bc4-3333-4866-8ca3-e69d8e8e4959 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.643154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 59/122] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:14.643653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b6bc4-3333-4866-8ca3-e69d8e8e4959 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.649933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.651001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.651502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.651944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.669044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.671509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-37196224-0e0a-4669-abd7-a926b6de3540 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:14.681727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:14.682880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:14.683643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:14.684924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.685224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.685603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.696113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:14.717296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:14.718498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:14.792436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b6bc4-3333-4866-8ca3-e69d8e8e4959 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:14.816923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 65/124] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:14.817942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.832441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:14.833799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:14.834307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:14.834867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:14.836417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:14.838149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:14.840171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 60/125] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:14.847546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:14.963431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4830c46-4125-4976-8952-0a3e49c4c114 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.000909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.013656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.014185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.015474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.015691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.016435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:15.016745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.018130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.019779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.038277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 66/126] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:15.038976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.041989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99c66-e891-4af2-8cdb-c34801c5d608 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.111877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.112665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.112947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.113220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.114472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.115932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.117638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 61/127] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:14 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:15.118473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.218359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b02596-dfb3-40e3-b92f-8de499975678 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.258049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31ddd-1e5d-42f8-affc-0ca57305bc01 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.295423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.296374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.296660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.297639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] PATH_INFO: `/v3/users/1f1a77a7f16346f3824cb61768746554` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.303689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.304504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.304796 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.305074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.312266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] RBAC: Authorizing `identity:get_user(user_id=1f1a77a7f16346f3824cb61768746554)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.313865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.315612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/users/1f1a77a7f16346f3824cb61768746554 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:15.316357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.327249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.339239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:15.355626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:15.356532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:15.357330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 67/129] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:15.358090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.425596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955429a8-1d4e-41a9-a2fb-236ec0f3016a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.466196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.467159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.470352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.470352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.474827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, user_id=5d213696814d4a9aaf5ee9a25ff03fa4, role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.476298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.493155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 63/130] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:15 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:15.494100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.514075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.572927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.619116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413d521-3cef-47f2-812e-f765789e008b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.622545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658381-800a-42e3-8afd-d01b472be8da None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.668691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.669386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.669676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.669954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.670745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] RBAC: Adding query filter params (name=member) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:15.672100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.673603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.705432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 64/131] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:15.707643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.708307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.708592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.708871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.712463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.842008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.843507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e347fa4-7eaa-4aa8-ba14-4cc4680d4484 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.887993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db419d33-982e-4959-a99b-4c9687b5e296 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:15.918918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:15.925286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:15.930441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:15.930441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:15.930441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:15.930441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:15.930441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:15.931889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 65/132] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:15.938552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:15.962794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:16.049884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 68/133] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:16.051052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235bb12-95a4-4fd6-84ee-dee37cad4d30 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.051359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235bb12-95a4-4fd6-84ee-dee37cad4d30 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.051624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235bb12-95a4-4fd6-84ee-dee37cad4d30 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.053155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 69/134] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:16.059945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430e82-db63-4637-8985-79930d53f03d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.060304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430e82-db63-4637-8985-79930d53f03d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.060593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b430e82-db63-4637-8985-79930d53f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.079948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8652ad-bdec-448a-a828-8c2945fcd1fb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.092842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2b430e82-db63-4637-8985-79930d53f03d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:16.094305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2b430e82-db63-4637-8985-79930d53f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:16.119843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.120741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.121031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.121317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] PATH_INFO: `/v3/projects/9af495954bc243e797f6e697f4005461` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.126439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] RBAC: Authorizing `identity:get_project(project_id=9af495954bc243e797f6e697f4005461)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:16.131115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:16.138669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 66/135] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:15 2022] GET /identity/v3/projects/9af495954bc243e797f6e697f4005461 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:16.139546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:16.213494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 70/136] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:16.214250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b430e82-db63-4637-8985-79930d53f03d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:16.286701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:16.331724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:16.347426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b430e82-db63-4637-8985-79930d53f03d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.376857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ea40-2237-4ff8-899d-cf7e15ee74af None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.389367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.391950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.393080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.393568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] PATH_INFO: `/v3/projects/9af495954bc243e797f6e697f4005461/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.397266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=9af495954bc243e797f6e697f4005461, user_id=5d213696814d4a9aaf5ee9a25ff03fa4, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:16.400674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:16.427226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 71/137] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:16 2022] PUT /identity/v3/projects/9af495954bc243e797f6e697f4005461/users/5d213696814d4a9aaf5ee9a25ff03fa4/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:16.428644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:16.431487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.432668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.433008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.433296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.442428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:16.444057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:16.446216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 67/138] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:16 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:16.452539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:16.661932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a75314-9185-4a9c-8b3d-ffdb82b20b87 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.678460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb25e1-df81-4bc8-8f1c-e0d494685b32 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.725141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.725813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.726102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.726403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.727255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=1f1a77a7f16346f3824cb61768746554) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:16.727575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:16.729050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:16.731178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:16.739142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 68/139] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:16 2022] GET /identity/v3/role_assignments?user.id=1f1a77a7f16346f3824cb61768746554&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:16.750951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216473-0916-4442-b030-85773ad55848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.751771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216473-0916-4442-b030-85773ad55848 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.752063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216473-0916-4442-b030-85773ad55848 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.752346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216473-0916-4442-b030-85773ad55848 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.776350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f216473-0916-4442-b030-85773ad55848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6f216473-0916-4442-b030-85773ad55848 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:16.789082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:16.790042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:16.790845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 72/140] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:16.796466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:16.927595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9b16a-8197-4012-b3b2-344edbeeafe1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:16.978112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:16.978819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:16.979110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:16.979393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:16.980144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] RBAC: Adding query filter params (name=member) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:16.980507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:16.981888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231d53-6908-422d-8d7e-10036fd1ea08 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:16.994366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 69/141] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:17.000566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216473-0916-4442-b030-85773ad55848 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:17.018300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:17.163530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216473-0916-4442-b030-85773ad55848 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:17.208307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:17.209563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.210121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.210634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.215069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:17.215706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf4a-6952-445f-b484-caf4ec947a71 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.216001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf4a-6952-445f-b484-caf4ec947a71 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.216283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf4a-6952-445f-b484-caf4ec947a71 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.217412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a684143-2909-434d-a8e6-c2947b8b6378 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:17.218455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 70/142] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:17.220097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 73/143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:16 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:17.225408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7cf4a-6952-445f-b484-caf4ec947a71 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:17.230287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5f86-38aa-4242-b538-d01c5b9152fa None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.230691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5f86-38aa-4242-b538-d01c5b9152fa None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.230976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5f86-38aa-4242-b538-d01c5b9152fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.256091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c97e5f86-38aa-4242-b538-d01c5b9152fa None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:17.257614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c97e5f86-38aa-4242-b538-d01c5b9152fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:17.357316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:17.398433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 74/144] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:17.409868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98904ed-7edf-4cdf-9293-778536e320e7 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.410542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98904ed-7edf-4cdf-9293-778536e320e7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.411026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98904ed-7edf-4cdf-9293-778536e320e7 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.418451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 75/145] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:17.423909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.424530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.425010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.438347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:17.456520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:17.459037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:17.470481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7cf4a-6952-445f-b484-caf4ec947a71 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:17.530771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:17.531857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.532414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.533010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.537771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 76/146] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:17.539049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:17.656460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:17.657947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:17.688775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2df04-6a74-4086-b9f5-d8c18c3880cd None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:17.743629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:17.744350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:17.744633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:17.744907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:17.751280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:17.752818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:17.754540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 77/147] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:17.755316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:17.763113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:17.777157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:17.848063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 71/148] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:17.848777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:17.958662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ebd89-5115-4f60-b172-a185c397ddcb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.013923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c687fb4-62a2-4bb5-aebb-a66e359a8400 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.017180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.017857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.018143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.018442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.020060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.021512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.061397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 72/149] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:17 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:18.062220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.090443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.091327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.091610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.091884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.102212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:18.112593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:18.120810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:18.121737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:18.122981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 78/150] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:18.129420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.242134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:18.263768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc835e0c-4423-4705-ac08-799cb82370f6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.285906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad00fe7-86ca-4a1a-afee-a62f238d0db0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.321999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.322706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.322996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.323277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.324303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:18.324614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.335249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 79/151] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:18.337815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.338715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.339001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.339278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.340493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.341464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.352654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.354762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 73/152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:18.361135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.472527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101032e3-761d-41d0-8cc8-bc24c9cea953 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.486536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:18.497285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:18.506058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa71c0-520a-4416-b2c0-1f8a31f0a0fe None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.524454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.525343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.525629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.525908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] PATH_INFO: `/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.531058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] RBAC: Authorizing `identity:get_user(user_id=5c7b0226fd854776bbee2a2d0d0ee93a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.533089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.534797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 80/153] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/users/5c7b0226fd854776bbee2a2d0d0ee93a => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:18.547608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.550198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.551206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.551681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.552198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.553382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:18.553920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.555573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7fe1a-38c4-4b7a-880e-beecacf3430b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.562142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 74/154] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:18.671540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224721f-b0ed-4930-b449-ff841eebf5ea None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:18.675466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ab25-7931-4a06-81e5-b100835580f6 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.676004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ab25-7931-4a06-81e5-b100835580f6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.676442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ab25-7931-4a06-81e5-b100835580f6 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.679274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 75/155] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:18.683555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.684089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.684542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.715894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:18.717411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:18.760262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:18.760462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:18.761061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:18.761061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:18.827267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:18.837432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:18.839491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:18.842134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 76/156] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:18.871472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:18.945165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 81/157] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:18.950960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.001794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.032321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5497ed8-064f-41ba-bf64-0d4ed2fb56c0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.063709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.076861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34f8e8-3547-49df-9ca1-0f094171ca9e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.079846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.081582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.081904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.082169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.102314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:19.114909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:19.115922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:19.116711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 77/158] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:19.123967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.207068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.207951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.208238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.208519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.211105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.212606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.214384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 82/159] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:18 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:19.220353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.238228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80097279-3f51-4e15-a2d9-dc91b50d0eba None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.278658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.279343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.279627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.279901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.280656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:19.280962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.282440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.292645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 78/160] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:19.294697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.299518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.382726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf65a8-81fe-4271-80d5-844d4c691256 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.421157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.423694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.424201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.424681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.425904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5c7b0226fd854776bbee2a2d0d0ee93a) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:19.426456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.428110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da249b-5a99-4c99-8edd-3007f6b204c4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.428652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e02d8-8b8b-4c02-8d15-cbdf14581af9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.435443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 83/161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/role_assignments?user.id=5c7b0226fd854776bbee2a2d0d0ee93a&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:19.438817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149bf8-a3b8-46df-8059-48657aa48d7b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.439308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149bf8-a3b8-46df-8059-48657aa48d7b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.439901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149bf8-a3b8-46df-8059-48657aa48d7b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.445747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 84/162] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:19.448081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.448597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.449392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.472710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:19.475185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:19.476622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.477680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.478128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.478494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.480233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.481461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.483663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 79/163] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:19.489844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.498770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.563396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.563816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 85/164] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:19.585108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.600107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb321-5f6d-455e-a689-d16578c1a22a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.663262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.664027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.664413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.665419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.699069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cb247-a896-4d42-a452-31b9822aaf9e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.737704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.739047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.739490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.739903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.751839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-19e311bc-d32b-4eda-8b95-85fb489ba71e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:19.760387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:19.761421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:19.762476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:19.765075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 86/165] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:19.771965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560e23-5adb-4d22-88b9-9ddb26637f95 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.772456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560e23-5adb-4d22-88b9-9ddb26637f95 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.772886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560e23-5adb-4d22-88b9-9ddb26637f95 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.774456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 87/166] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:19.776204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560e23-5adb-4d22-88b9-9ddb26637f95 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.778598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.780170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f11775-c396-41df-bf28-33b57b3b12d0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.810765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.884853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:19.896639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560e23-5adb-4d22-88b9-9ddb26637f95 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:19.933529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 80/167] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:19.934204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:19.935251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.935731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.936201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.937140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:19.937766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:19.939563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:19.943790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:19.944081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:19.944362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:19.950681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 88/168] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:19.955335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:19.975367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:19.976583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:20.062852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.069726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfd8a7-2015-4e4c-ab31-37de40031ca8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.098247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 81/169] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:20.098934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.110071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.111332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.111917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.112363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.113845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.115701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.117629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 89/170] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:19 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:20.124879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.214802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.217024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a8861-7296-43f6-8c10-1818e1025a7d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.254239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.256536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.256536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.256536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] PATH_INFO: `/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.257233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] RBAC: Authorizing `identity:get_user(user_id=5d213696814d4a9aaf5ee9a25ff03fa4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.258856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.260636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 82/171] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/users/5d213696814d4a9aaf5ee9a25ff03fa4 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:20.265031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.301450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d9b-4cd6-465d-a35d-1a7773ad7e2b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.374350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.375672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.375955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.376228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.377066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:20.377379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.377878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af3d16-3b81-4a5b-b20a-f712ed2202eb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.378850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.396488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 90/172] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:20.397429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.417101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.417448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.418692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.420442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.420442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.433207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:20.444362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:20.445534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:20.446840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 83/173] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:20.452935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.495210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.582294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8ef77-41a7-46c0-a7dd-7a84f95d5e93 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.620484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700e3e-da03-4dc4-9201-a2e0f7c78b29 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.621133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.621432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.621718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.622177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.623036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:20.623391 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.625380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.636327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 84/174] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:20.642316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.667739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.668810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.669318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.669799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.745804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.747630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.761647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.767079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:20.776305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7312af6-5a37-49e5-a768-221499447ec4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.818738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:20.819664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:20.819948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:20.820225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] PATH_INFO: `/v3/users/1f1a77a7f16346f3824cb61768746554` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:20.827539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] RBAC: Authorizing `identity:get_user(user_id=1f1a77a7f16346f3824cb61768746554)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:20.828756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:20.830727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 85/175] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/users/1f1a77a7f16346f3824cb61768746554 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:20.835790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.861985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 91/176] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:20.867938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:20.972568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c88cf-a0b4-4d72-949d-050d472b2129 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:20.982479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:21.009393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:21.010182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.010491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.010764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] PATH_INFO: `/v3/projects/9af495954bc243e797f6e697f4005461` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.011943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] RBAC: Authorizing `identity:get_project(project_id=9af495954bc243e797f6e697f4005461)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:21.013408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:21.019974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 92/177] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/projects/9af495954bc243e797f6e697f4005461 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:21.021136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:21.028462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2afb-9fc6-4994-9dbf-b9ec63ca7a6e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:21.108167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:21.110005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.110541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.112547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:21.114266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:21.116742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 86/178] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:20 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:21.124534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:21.150482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857474e-3816-4acb-b58c-121fec29139e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:21.170454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:21.184625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:21.185285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.185567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.185843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.187154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=9af495954bc243e797f6e697f4005461, user.id=5d213696814d4a9aaf5ee9a25ff03fa4) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:21.187154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:21.188542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0db13d-5e50-48c7-9a87-bd7e41f81641 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:21.200951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 93/179] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:21 2022] GET /identity/v3/role_assignments?user.id=5d213696814d4a9aaf5ee9a25ff03fa4&scope.project.id=9af495954bc243e797f6e697f4005461&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:21.299563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe53521-dcfb-4ee7-beab-21c7724a4e78 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:21.334720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:21.335657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.335934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.336201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/1f1a77a7f16346f3824cb61768746554/roles/45bbda86ddc84212b101d67cae3f62ca` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.338265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, user_id=1f1a77a7f16346f3824cb61768746554, role_id=45bbda86ddc84212b101d67cae3f62ca)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:21.339747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3c6c1-a3a3-4205-a5d4-07f1790629ca None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:21.367385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 87/180] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:21 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/users/1f1a77a7f16346f3824cb61768746554/roles/45bbda86ddc84212b101d67cae3f62ca => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:21.771266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53bb42-3c94-404b-8550-a47dd6822969 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.771766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53bb42-3c94-404b-8550-a47dd6822969 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.771766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53bb42-3c94-404b-8550-a47dd6822969 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.773201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 94/181] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:21.778393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a19c38-1f82-4b27-b9fe-be39325bfb5e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:21.778784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a19c38-1f82-4b27-b9fe-be39325bfb5e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:21.778968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a19c38-1f82-4b27-b9fe-be39325bfb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:21.796265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-46a19c38-1f82-4b27-b9fe-be39325bfb5e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:21.797519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-46a19c38-1f82-4b27-b9fe-be39325bfb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:21.865474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 88/182] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:21.889457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53bb42-3c94-404b-8550-a47dd6822969 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:21.934567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:21.996284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53bb42-3c94-404b-8550-a47dd6822969 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.013916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335eda2-3ce0-445a-8060-cde73a2234e5 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.014297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335eda2-3ce0-445a-8060-cde73a2234e5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.014695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335eda2-3ce0-445a-8060-cde73a2234e5 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.016879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 89/183] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:22.021743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.021968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.022251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.034915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.036347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.036842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.037318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.040165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:22.041365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:22.049925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:22.053945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:22.061502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:22.063036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:22.063822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:22.066572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 95/184] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:22.072236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.123314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 90/185] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:22.149548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.188905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.191856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837228f6-005d-4fa5-ba67-b8e5d67be625 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.227252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.227924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.228212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.228491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.229236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:22.229546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.230954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4a6d-4704-4a01-b3a6-a0d28fd63345 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.243860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 96/186] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:22.243860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e28cb6-340c-4735-b19b-fea6c3fbe9e9 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.243860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e28cb6-340c-4735-b19b-fea6c3fbe9e9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.243860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e28cb6-340c-4735-b19b-fea6c3fbe9e9 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.243860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 97/187] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:22.247560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e28cb6-340c-4735-b19b-fea6c3fbe9e9 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.263414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616ecd-0930-4df0-a432-30c9f01a8a4c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.299384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.300180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.300465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.300739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.301898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.303388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c4276-fb53-4f29-9dfe-2b2e72dec95e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.309371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 91/188] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:22.309371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.309371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.309371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.328076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:22.329792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:22.339048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.363754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e28cb6-340c-4735-b19b-fea6c3fbe9e9 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.398376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.399245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.399528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.399805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.402892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 92/189] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:22.407204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.414601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.416151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.422844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 98/190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:22.424305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.448819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.517621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf9fc-872b-4747-8127-9fa6fc535914 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.539474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4b533-4f37-4c63-86e5-479140168d9f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.554325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.555012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.555293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.555561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.556372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.557923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.573331 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.576641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.577408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.577703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.578066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.586771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 93/191] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:22 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:22.588084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.695558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.697164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.722749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.728531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c869-290c-4a1b-8ca0-baacc1aaaa93 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.774223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.775158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.775441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.775719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.791402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:22.801067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:22.801954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:22.802817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:22.803429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 94/192] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:22.809526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.833297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 99/193] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:22.840236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:22.881891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:22.937890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1153-1cc1-4c4f-8888-31ce5913f4b6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.967976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4854e-db66-4280-a1a1-d91a86c75e76 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:22.980910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:22.981983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:22.982273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:22.982644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:22.983410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:22.984088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:22.986036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:22.995284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 95/194] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:22.996390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:23.000712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:23.012554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.013415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.013695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.013970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.015245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.017236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.019052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 100/195] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:23.028733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:23.136006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b847f8e-71c0-404e-978d-c218b5c22660 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:23.155663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191978-5ff1-499a-b50c-b94498d20df9 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:23.159218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:23.175534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.176736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.177194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.177617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.179045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.180715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.183180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 96/196] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:22 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:23.192996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:23.201397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.202196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.202582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.210354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.210354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=e0cbc34b15e04f1aaa2a66b0e8dead70) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:23.210354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.210354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.226633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 101/197] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:23 2022] GET /identity/v3/role_assignments?user.id=e0cbc34b15e04f1aaa2a66b0e8dead70&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:23.355155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac110c4-86a0-4d51-983d-673cb016a40f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:23.392301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.393191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.393645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.394075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.454717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:23.467654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.469600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.574683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 97/198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:23.583569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:23.660071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:23.715894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128072-92d5-41d6-a2e5-052ce1e735f4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:23.751912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.752782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.753068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.753343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.754567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.756018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c46cd4-46e4-41e5-b89d-fc2d51968070 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.757723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 102/199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:23 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:23.763695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:23.826606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249be07b-6256-4dd7-800c-c91fd5b732fb None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.826931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249be07b-6256-4dd7-800c-c91fd5b732fb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.827173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249be07b-6256-4dd7-800c-c91fd5b732fb None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.828863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 103/200] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:23.834125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.834699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.834699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.851589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:23.852792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:23.869438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:23.882071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf095f-f8e7-4bac-9065-dc5a1287340c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:23.924451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:23.925841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:23.926120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:23.926412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:23.927239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=e0cbc34b15e04f1aaa2a66b0e8dead70) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:23.927547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:23.928981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:23.943247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 98/201] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:23 2022] GET /identity/v3/role_assignments?user.id=e0cbc34b15e04f1aaa2a66b0e8dead70&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:23.958107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 104/202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:23.986252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:24.129910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94800f72-34a2-46cb-aaf6-1f3a3f5e024d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:24.175783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:24.176520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:24.176801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:24.177071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:24.189189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:24.197029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:24.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:24.199163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 99/203] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:24.204427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:24.265105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:24.338381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb72b87-560d-43de-b3a4-3da521b30dc8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:24.384246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:24.384898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:24.385182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:24.385458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:24.386194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] RBAC: Adding query filter params (name=reader) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:24.386532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:24.387899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:24.401430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 105/204] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:24.407743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:24.423837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:24.525130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958b1f2-6153-4d06-9255-2edba0ce8e97 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:24.558741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:24.559612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:24.559898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:24.560171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] PATH_INFO: `/v3/users/1f1a77a7f16346f3824cb61768746554` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:24.561362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] RBAC: Authorizing `identity:get_user(user_id=1f1a77a7f16346f3824cb61768746554)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:24.562880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:24.564726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 100/205] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:24 2022] GET /identity/v3/users/1f1a77a7f16346f3824cb61768746554 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:24.570214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:24.721473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6be4ca-bf72-4c5e-b02b-3011d17bc3df None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:24.750837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:24.765735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:24.766753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:24.767247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:24.767723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:24.839409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:24.841683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:24.944412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 106/206] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:24.950228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:25.064145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:25.087024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9a0e1-e108-4a1a-aab0-48291513f204 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:25.120791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:25.121978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.122588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.123080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.125133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:25.126899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a75006-0cfd-44f1-8f08-b1d2671650db None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:25.128816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 101/207] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:24 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:25.134680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:25.162355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:25.254429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247534-1d61-4d68-b8ba-e204458b85b6 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:25.296385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:25.297924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.298434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.298910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.299947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c, user.id=1f1a77a7f16346f3824cb61768746554) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:25.300486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:25.302158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21e467-9498-44d7-8528-d2f478b364e2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:25.308842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 107/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:25 2022] GET /identity/v3/role_assignments?user.id=1f1a77a7f16346f3824cb61768746554&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:25.371826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671710-7da5-4d72-8b5f-94bcc187b6b7 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.372443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671710-7da5-4d72-8b5f-94bcc187b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.372948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671710-7da5-4d72-8b5f-94bcc187b6b7 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.375218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 102/209] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:25.388225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286656e3-6e9c-414a-abfe-656b0e7c90bf None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.388857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286656e3-6e9c-414a-abfe-656b0e7c90bf None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.389340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286656e3-6e9c-414a-abfe-656b0e7c90bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.413400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-286656e3-6e9c-414a-abfe-656b0e7c90bf None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:25.415019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-286656e3-6e9c-414a-abfe-656b0e7c90bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:25.507633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 108/210] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:25.535561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671710-7da5-4d72-8b5f-94bcc187b6b7 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:25.622412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b65bf-6063-4492-9c80-ba1b4d3ad427 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.622700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b65bf-6063-4492-9c80-ba1b4d3ad427 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.622931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b65bf-6063-4492-9c80-ba1b4d3ad427 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.624475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 109/211] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:25.630828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038ce9-a242-4cb4-b73d-35a8bf58763c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.631116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038ce9-a242-4cb4-b73d-35a8bf58763c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.631396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038ce9-a242-4cb4-b73d-35a8bf58763c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.642518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:25.649241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-db038ce9-a242-4cb4-b73d-35a8bf58763c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:25.650457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-db038ce9-a242-4cb4-b73d-35a8bf58763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:25.685084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671710-7da5-4d72-8b5f-94bcc187b6b7 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:25.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:25.723577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.724077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.724580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.726161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:25.727956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:25.729917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 103/212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:25.735859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:25.738496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 110/213] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:25.754148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:25.764480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe663b8-ecae-4171-8424-d381179d1d4e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.764784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe663b8-ecae-4171-8424-d381179d1d4e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.765035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe663b8-ecae-4171-8424-d381179d1d4e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.766809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 111/214] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:25.768244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe663b8-ecae-4171-8424-d381179d1d4e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:25.846452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c8b93-6f54-4296-a001-c0a480d96e4c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:25.876995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe663b8-ecae-4171-8424-d381179d1d4e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:25.880809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:25.881480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.881764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.882037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.882773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:25.884214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a25b6-e817-42f6-85f9-b56522ff2544 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:25.905199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 104/215] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:25 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:25.912517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.913237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.913580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:25.914033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.914423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:25.914706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:25.914979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:25.918722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:25.928996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:25.938990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:25.939951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:25.940761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 112/216] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:25.942725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:25.944252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:25.946175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.063114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 105/217] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:26.069430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe65df-3ecb-428c-ba59-9db6d5f035f2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.086116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.088444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.109479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.110145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.110483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.110758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.111504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:26.111808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.113406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.122705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 113/218] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:26.128811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.196937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.211405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c547a5d-4284-46cd-b33f-7cc5ed7829d3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.250990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f012ae-94cf-4d96-847b-430554bbc0f3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.251315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.252126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.252477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.252869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.266211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:26.274022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:26.274980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:26.275773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 106/219] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:26.282763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.299693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276366-7930-4004-8549-b524ac73a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.300714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276366-7930-4004-8549-b524ac73a18b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.301003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276366-7930-4004-8549-b524ac73a18b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.301280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93276366-7930-4004-8549-b524ac73a18b None admin] PATH_INFO: `/v3/users/3fdc0faf605240f2a80d07961f658937` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.323978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93276366-7930-4004-8549-b524ac73a18b None admin] RBAC: Authorizing `identity:get_user(user_id=3fdc0faf605240f2a80d07961f658937)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.325456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93276366-7930-4004-8549-b524ac73a18b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.327656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 114/220] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/users/3fdc0faf605240f2a80d07961f658937 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:26.333438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276366-7930-4004-8549-b524ac73a18b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.367913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.412314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a2643-306d-48d4-b104-f933e41375a9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.449608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.450360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.450647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.450919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.451667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:26.451971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.453421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.455078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93276366-7930-4004-8549-b524ac73a18b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.462546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 107/221] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:26.467758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.488203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.542870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.543615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.544002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.544279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.577873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29e346-ea96-4a80-8696-f1b8cbeb92b2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.601746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.615141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.616653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.616945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.617361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.618959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.621126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1974f1-b19a-48bd-a0bb-e397cb7d74c9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.623138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 108/222] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:26.624408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aee68-8a24-4e45-97e9-28d11fdf9396 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.624729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aee68-8a24-4e45-97e9-28d11fdf9396 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.624999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217aee68-8a24-4e45-97e9-28d11fdf9396 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.630407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 109/223] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:26.631284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aee68-8a24-4e45-97e9-28d11fdf9396 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.634315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.635826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7ccd-6f81-4c24-b551-4b51972da8eb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.746209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 115/224] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:26.748034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.748547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.749082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.753284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217aee68-8a24-4e45-97e9-28d11fdf9396 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.767731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:26.769423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:26.786740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.789939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.790853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.791143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.791418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.792622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.794073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.796002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 110/225] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:26.796816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.856428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 116/226] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:26.856428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.869939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:26.925345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb58a4-eb91-41db-9015-60af160ef47a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:26.968969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:26.970570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:26.971215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:26.971788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:26.973377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:26.975357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:26.977312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 111/227] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:26.978914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:26.997059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e07f81-e6a7-47b5-8cb6-1cf1dc3cd7fc None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.035327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.036252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.036537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.036812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/e0cbc34b15e04f1aaa2a66b0e8dead70/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.038862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, user_id=e0cbc34b15e04f1aaa2a66b0e8dead70, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.040290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.057449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 117/228] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:26 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/e0cbc34b15e04f1aaa2a66b0e8dead70/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:27.058085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:27.068624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:27.090726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737afe6e-1a4b-4e79-8c83-e9be82600398 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.120223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:27.129559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.130715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.131193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.131660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.143814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:27.152636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:27.153556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:27.154367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:27.162647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 112/229] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:27.164356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:27.179545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab318cd9-1adb-4590-8e3e-8e664f957499 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.220958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.221801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.222233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.222857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.223941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=3fdc0faf605240f2a80d07961f658937) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:27.224257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.225778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.232496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 118/230] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:27 2022] GET /identity/v3/role_assignments?user.id=3fdc0faf605240f2a80d07961f658937&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:27.282532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8061b7-9179-4d92-991a-ec13f95af354 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.321638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.322707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.323186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.323912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.324891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:27.325410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.327091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.337430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 113/231] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:27.343574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:27.422017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:27.460043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ea958-be85-43f5-9863-4a970371a9ef None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.506530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.507407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.507688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.507959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.509157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.510617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.512259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 119/232] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:27 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:27.518454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:27.573776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:27.624366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36222a98-cb09-4d60-a5e7-dd1b2801bb84 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.658282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.659170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.659449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.659720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.660909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.662359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d70d5-e8da-46b4-89d7-0216a500c7cd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.664067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 114/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:27 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:27.669848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:27.789239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ef150-0b97-4339-92fc-5a7f469d6910 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:27.825641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15960342-5203-4187-9663-62a614d2725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:27.826476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960342-5203-4187-9663-62a614d2725f None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:27.826758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960342-5203-4187-9663-62a614d2725f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:27.827033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15960342-5203-4187-9663-62a614d2725f None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/e0cbc34b15e04f1aaa2a66b0e8dead70/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:27.829803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15960342-5203-4187-9663-62a614d2725f None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, user_id=e0cbc34b15e04f1aaa2a66b0e8dead70, role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:27.831264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15960342-5203-4187-9663-62a614d2725f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:27.860959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:27.866665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 120/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:27 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/e0cbc34b15e04f1aaa2a66b0e8dead70/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:28.641542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a505-84c3-4716-b268-063ebcf97032 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:28.642060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a505-84c3-4716-b268-063ebcf97032 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:28.642060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a505-84c3-4716-b268-063ebcf97032 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:28.643499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 115/235] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:28.648800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:28.649191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:28.649371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:28.670036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:28.671337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:28.713935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e00c-60da-4442-96dc-4c8c7a11aaa6 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:28.714583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e00c-60da-4442-96dc-4c8c7a11aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:28.715059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e00c-60da-4442-96dc-4c8c7a11aaa6 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:28.716641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 116/236] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:28.723233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:28.724203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:28.724682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:28.745146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:28.746866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:28.776987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 121/237] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:28.824894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:28.868807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 117/238] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:28.913345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:28.952155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb2f9-99c6-4499-ac1f-2472f6980add None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:28.980699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:28.993758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:28.994988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:28.995472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:28.995963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.003046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:29.008724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:29.013301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:29.015802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 122/239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:29.022627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:29.086679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ebda-bed3-47bf-a2c0-5973ff2162bb None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:29.146049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:29.146878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.147163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.147437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.170068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:29.177709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:29.178629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:29.179405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 118/240] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:29.184638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:29.218136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b799c8-c36c-4721-84d5-3423f5fd7371 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:29.282275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:29.285095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.285095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.285095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.285490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:29.287053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:29.317145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 123/241] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:29 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:29.357317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ce22a-c0ff-40ab-86da-0f2340268215 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:29.377306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:29.411908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:29.412702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.412997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.413278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.414062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:29.414406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:29.415908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:29.431629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 119/242] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:29.436708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:29.525056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:29.600857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f2116-71b6-49f3-85ea-f75b3c1468cf None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:29.648322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:29.649545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.650478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.650947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] PATH_INFO: `/v3/users/b254f1f98a93462b91607aea8732f76b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.673568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b254f1f98a93462b91607aea8732f76b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:29.675394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98aec3f-169d-4442-83c5-b242b3cca7c2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:29.677346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 124/243] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:29 2022] GET /identity/v3/users/b254f1f98a93462b91607aea8732f76b => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:29.683106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:29.710085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5f1-ebeb-471b-bb0f-c3130ee0984f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.710760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5f1-ebeb-471b-bb0f-c3130ee0984f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.711230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5f1-ebeb-471b-bb0f-c3130ee0984f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.712892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 125/244] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:29.716618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3369-d555-491f-8d2e-1840158281b0 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.717138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3369-d555-491f-8d2e-1840158281b0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.717421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3369-d555-491f-8d2e-1840158281b0 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.718979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 126/245] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:29.725982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb156c8-2c90-4b46-8654-3fd3c8474fb8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.726271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb156c8-2c90-4b46-8654-3fd3c8474fb8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.726586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb156c8-2c90-4b46-8654-3fd3c8474fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.745103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4bb156c8-2c90-4b46-8654-3fd3c8474fb8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:29.746579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4bb156c8-2c90-4b46-8654-3fd3c8474fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:29.820056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55f29-cc4d-4e24-bbf1-c0576b5c9665 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:29.834954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 127/246] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:29.841872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.842474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.842760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.862676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:29.863209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:29.863473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:29.863799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:29.864139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:29.864651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:29.908412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:29.911943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:29.946976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 128/247] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:29.951022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:29.951233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:29.952537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.096475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87351a4d-8916-4f43-8595-eeb00ba99e33 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.152533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 120/248] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:30.153882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.164062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.164877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.165165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.165564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.194856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:30.203401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:30.204383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:30.205210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:30.207344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 129/249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:30.208414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.258015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.266075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.312116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56b9a-4e08-42a8-b243-9dbcc9a3fa84 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.348250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cefa0-bb38-4753-9150-e84e7a5f4cba None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.360092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.360912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.361192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.361463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.379935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:30.385591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.386779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.387294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:30.387773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:30.388669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:30.389486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 121/250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:30.390196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.390943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.392484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.398993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.401462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 130/251] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.402673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.517385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688d3e-4d03-4d32-9b1d-26ddeac44c7e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.547054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.554200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.554917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.555197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.555470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.556209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:30.556513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.557887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.569679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 122/252] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.570548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.571627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.576289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884fe45-eb85-425c-96bf-377f3fb4e89a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.613669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.614795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.615379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.615912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.616849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:30.617366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.619191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d4057-37ba-46d4-b799-f7e834ca4d0c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.628995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 131/253] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.630868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cd2b-bda6-4262-bab7-4cc3f06ae097 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.631426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cd2b-bda6-4262-bab7-4cc3f06ae097 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.631904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9cd2b-bda6-4262-bab7-4cc3f06ae097 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.637533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 132/254] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:30.638937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9cd2b-bda6-4262-bab7-4cc3f06ae097 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.672968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccc40f-3d43-4cd6-bbaa-711eafb2a1c5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.707944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.708676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.708899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.709181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.710209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=b254f1f98a93462b91607aea8732f76b) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:30.710448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.711884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.717886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 123/255] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/role_assignments?user.id=b254f1f98a93462b91607aea8732f76b&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.719170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.756835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9cd2b-bda6-4262-bab7-4cc3f06ae097 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.781996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.797695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.799016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.799528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.800003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.801472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.803307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4bcf-1575-4c53-acf0-389e4d8f0462 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.805386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 133/256] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.807204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.807720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.808205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.825568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:30.827277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:30.828947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527f29-14dd-4a2b-a9cb-8887dfc747fb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:30.853898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:30.874749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:30.875646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:30.875925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:30.876199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] PATH_INFO: `/v3/users/3fdc0faf605240f2a80d07961f658937` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:30.877641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=3fdc0faf605240f2a80d07961f658937)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:30.879138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:30.880834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 124/257] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/users/3fdc0faf605240f2a80d07961f658937 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:30.881626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:30.952270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 134/258] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:30.958964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.103991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f34e5-fb0f-4252-b59c-7748b5a71fd4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.146722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:31.155146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef0ff9-abc2-438c-a04f-9db0b1cda53f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.169004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.170013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.170521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.171881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.215620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.216863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.217348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.217823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.219256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:31.220945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:31.223245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 135/259] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:31.224210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.255233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:31.287533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:31.289736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:31.404015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 125/260] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:31.405264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.438708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:31.449508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7211e5-ef12-4de7-97f9-b7a6722b0e44 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.529346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b23cf-82a6-4176-9ca0-30e22c8f5aa4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.559637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.560464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.560746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.561019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.569242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.570144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.570468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.570745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/3fdc0faf605240f2a80d07961f658937/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.572792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, user_id=3fdc0faf605240f2a80d07961f658937, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:31.574567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:31.590013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 126/261] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:31 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/3fdc0faf605240f2a80d07961f658937/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:31.594703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.603334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:31.620603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:31.621666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:31.622923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 136/262] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:31.631433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.809627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:31.839074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d41790-068f-444a-9214-290ed14bc2a2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.854284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:31.876721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e0a87-9c9c-4f6f-94ee-222ec0658f3c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:31.880690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.881792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.882275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.882805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.884221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:31.885925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:31.887976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 127/263] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:31 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:31.895286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:31.934833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:31.935515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:31.935801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:31.936081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:31.936884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:31.937195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:31.938640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:31.948281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 137/264] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:31.955258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.146982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa9f9a-8aa7-490d-a9f6-b6d8971ee392 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.209213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb5bba-72e5-490a-9861-8689c751ddf4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.216997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:32.217661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.217971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.218799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.219635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=e0cbc34b15e04f1aaa2a66b0e8dead70) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:32.219948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:32.221546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6482b-fe5b-438e-9067-881673d685f7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:32.233878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 128/265] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:31 2022] GET /identity/v3/role_assignments?user.id=e0cbc34b15e04f1aaa2a66b0e8dead70&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:32.284416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf826ae-b681-4bb7-a5a6-2e4263a4747e None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.284736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:32.284882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf826ae-b681-4bb7-a5a6-2e4263a4747e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.284959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf826ae-b681-4bb7-a5a6-2e4263a4747e None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.285552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.285835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.286110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] PATH_INFO: `/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.286851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 129/266] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:32.291362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] RBAC: Authorizing `identity:get_user(user_id=e0cbc34b15e04f1aaa2a66b0e8dead70)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:32.292543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd81b1a-0f85-49c0-9e0d-765529155ce4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.292835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd81b1a-0f85-49c0-9e0d-765529155ce4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.292939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:32.293118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd81b1a-0f85-49c0-9e0d-765529155ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.294632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 138/267] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:31 2022] GET /identity/v3/users/e0cbc34b15e04f1aaa2a66b0e8dead70 => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:32.300960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.326279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-7bd81b1a-0f85-49c0-9e0d-765529155ce4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:32.328065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7bd81b1a-0f85-49c0-9e0d-765529155ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:32.338688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:32.350316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:32.481152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 130/268] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:32.488424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00a5b1-e774-443b-b052-d31c161b239e None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.488716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00a5b1-e774-443b-b052-d31c161b239e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.488994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00a5b1-e774-443b-b052-d31c161b239e None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.490578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 131/269] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:32.495186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.495470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.495743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.513319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:32.514623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:32.568126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfd2a2-06a8-400b-af3d-938b9ee7e04f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.584885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 132/270] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:32.585521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.607870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:32.608973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.609431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.609876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.690041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:32.691370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:32.693204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:32.718610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf1733-239b-409d-bcf8-5ef893fea83b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.719853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:32.760075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:32.760075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.760075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.760075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.775101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:32.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:32.787334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:32.788324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:32.790681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 133/271] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:32.792282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.838765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 139/272] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:32.839945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.928465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ec898-55d4-4643-99de-8b661d5f963c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.973707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a112a-47fb-4b9f-a40e-0a3caa2bb0f1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:32.977309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:32.978002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:32.978279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:32.978575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:32.983098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:32.988486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:32.988964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 134/273] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:32.989727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:32.995503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.000821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.021723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.022904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.023414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.023897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.024912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:33.025420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.027094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.036135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 140/274] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:33.037285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.103337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fcb52-88d8-46ec-b071-e7a780334512 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.141180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.142053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.142363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.142636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.144021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.145601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.147529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 135/275] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:32 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:33.148216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.152707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156bc1c-2273-4c00-b8b6-452715050f67 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.190135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.191330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.191838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.192322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.193299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.195182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.219862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 141/276] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:33 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:33.221295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.231559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.281854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.308483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c74a4-6ab3-4129-af56-23dbfe2414fd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.358481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.358758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bd12e-9ac8-4332-9d06-14873e05a584 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.359957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.360646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.362007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] PATH_INFO: `/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.380813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] RBAC: Authorizing `identity:get_user(user_id=4b6c7c9fc9d44dea9ad32b5395239bad)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.382646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea15e07-3a4d-4c0d-9d16-96e104b14555 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.384944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 136/277] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:33.386479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a4d9-dc3d-451a-b83d-f483b5b2b396 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.386964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a4d9-dc3d-451a-b83d-f483b5b2b396 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.387246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478a4d9-dc3d-451a-b83d-f483b5b2b396 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.388648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 137/278] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:33.391224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478a4d9-dc3d-451a-b83d-f483b5b2b396 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.403776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.404431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.404713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.405041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.406008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=e0cbc34b15e04f1aaa2a66b0e8dead70) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:33.406392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.407907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0027e-6b4b-4402-be65-853b5da425e9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.413826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 142/279] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/role_assignments?user.id=e0cbc34b15e04f1aaa2a66b0e8dead70&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:33.415038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.415323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.415600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.433998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:33.435346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:33.517079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478a4d9-dc3d-451a-b83d-f483b5b2b396 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.533542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 143/280] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:33.551212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.561529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.562847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.563509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.563509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.564320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, user.id=4b6c7c9fc9d44dea9ad32b5395239bad, scope.system=all) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:33.564494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.564874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.566549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.574491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 138/281] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/role_assignments?user.id=4b6c7c9fc9d44dea9ad32b5395239bad&scope.system=all&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:33.604362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:33.694208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cd609-79fd-47b4-8be8-64327e5d2d28 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.737133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.738310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.738835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.739307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.752479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:33.760736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:33.761630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:33.762455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:33.764437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 144/282] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:33.770840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:33.887835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bec04-3f50-4a09-9c0b-09ba90c1b510 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:33.925189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:33.925872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:33.926156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:33.926460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:33.927221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:33.927537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:33.929040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:33.938455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 139/283] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:33.943719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:34.079660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:34.092699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445bc56-6f16-4ecc-9b8c-ac60263bdf62 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:34.129885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:34.130808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.131093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.131374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] PATH_INFO: `/v3/users/b254f1f98a93462b91607aea8732f76b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.132609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] RBAC: Authorizing `identity:get_user(user_id=b254f1f98a93462b91607aea8732f76b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:34.134086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed2e5c-0ab5-43ae-9bfa-8064954e12aa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:34.140471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 145/284] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:33 2022] GET /identity/v3/users/b254f1f98a93462b91607aea8732f76b => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:34.141646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:34.211889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590f37-1d64-4d93-a9c5-3f2a67064bcc None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.212212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590f37-1d64-4d93-a9c5-3f2a67064bcc None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.212474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590f37-1d64-4d93-a9c5-3f2a67064bcc None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.214123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 146/285] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:34.219353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.219644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.219925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.237727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:34.238983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:34.241520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:34.329521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2139cf-2c68-42d9-81bb-31fced41c7d0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:34.352970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 147/286] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:34.383616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:34.384502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.384790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.385071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.386326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:34.387809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:34.389888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 140/287] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:34 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:34.395578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:34.418369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:34.589091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:34.632396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32483a2-e2d4-4594-999d-3e535dc50c63 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:34.635423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53c647-5240-4997-b327-57955b6ee32b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:34.661793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:34.683428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:34.684532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.684966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.685416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.694316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:34.695496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.695929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.697356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/b254f1f98a93462b91607aea8732f76b/roles/45bbda86ddc84212b101d67cae3f62ca` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.700864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, user_id=b254f1f98a93462b91607aea8732f76b, role_id=45bbda86ddc84212b101d67cae3f62ca)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:34.703464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:34.708991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:34.717343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:34.719020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:34.719827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:34.721823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 141/288] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:34.727876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:34.753638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 148/289] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:39:34 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/users/b254f1f98a93462b91607aea8732f76b/roles/45bbda86ddc84212b101d67cae3f62ca => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:34.841971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98eca-c0a3-413e-aff2-eb7e61419b14 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:34.882405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:34.883477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:34.883991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:34.884469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:34.885434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] RBAC: Adding query filter params (name=member) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:34.885950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:34.887683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:34.897796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 142/290] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:34.903980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:35.030025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b90ba-c750-44b0-a732-c42ccfb88c80 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:35.067907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:35.068783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:35.069120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:35.069391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] PATH_INFO: `/v3/users/3fdc0faf605240f2a80d07961f658937` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:35.070654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] RBAC: Authorizing `identity:get_user(user_id=3fdc0faf605240f2a80d07961f658937)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:35.072115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:35.074044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 149/291] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:34 2022] GET /identity/v3/users/3fdc0faf605240f2a80d07961f658937 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:35.081002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:35.134842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:35.193328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d2605-bf99-4d3b-a9bc-0a1d02c64d3d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:35.232221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:35.233185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:35.233841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:35.234468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:35.309536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:35.311354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:35.359204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:35.428310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 143/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:35.434054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:35.459435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:35.580576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361bfae-2fee-48f3-8900-32b01c8da4b5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:35.626416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:35.627289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:35.627577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:35.627851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:35.629057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:35.630536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d8a47-88a7-46a5-9334-32e1b8c2da19 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:35.632552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 150/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:35 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:35.638250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:35.785467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615473ce-530b-4cfc-8dd0-e1923b8a2fd8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:35.821656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:35.822750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:35.823238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:35.823977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:35.825007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=3fdc0faf605240f2a80d07961f658937) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:35.825524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:35.827318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933bb22-40c9-4633-b928-b5d9d6f44cb8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:35.834247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 144/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:35 2022] GET /identity/v3/role_assignments?user.id=3fdc0faf605240f2a80d07961f658937&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:36.044243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2abb22-f284-4fd3-b498-df768cfc41f3 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.044870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2abb22-f284-4fd3-b498-df768cfc41f3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.045362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2abb22-f284-4fd3-b498-df768cfc41f3 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.047732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 151/295] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:36.053905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bf13-8d19-4a66-a60d-aa45fdbe4fcb None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.054471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bf13-8d19-4a66-a60d-aa45fdbe4fcb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.054947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bf13-8d19-4a66-a60d-aa45fdbe4fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.074415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3541bf13-8d19-4a66-a60d-aa45fdbe4fcb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:36.075919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3541bf13-8d19-4a66-a60d-aa45fdbe4fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:36.080463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.161639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 145/296] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:36.184598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2abb22-f284-4fd3-b498-df768cfc41f3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.270403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314ca56-5d53-489b-b03a-ec6c27d5efce None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.270687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314ca56-5d53-489b-b03a-ec6c27d5efce None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.270977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314ca56-5d53-489b-b03a-ec6c27d5efce None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.272797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 146/297] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:36.279356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6ac7-5e54-48b7-9de5-ac26d2195035 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.279356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6ac7-5e54-48b7-9de5-ac26d2195035 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.279356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6ac7-5e54-48b7-9de5-ac26d2195035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.294473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.301436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e10a6ac7-5e54-48b7-9de5-ac26d2195035 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:36.303458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e10a6ac7-5e54-48b7-9de5-ac26d2195035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:36.317252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2abb22-f284-4fd3-b498-df768cfc41f3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.359149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.360096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.360479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.361579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.372590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:36.381070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:36.382206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:36.383266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:36.385026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 152/298] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:36.391422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.410414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 147/299] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:36.426037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f63b0-542c-4e58-984f-830575f77be6 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.426228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f63b0-542c-4e58-984f-830575f77be6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.426535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f63b0-542c-4e58-984f-830575f77be6 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.427898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 148/300] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:36.432302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f63b0-542c-4e58-984f-830575f77be6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.508117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.515474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1860d11-8827-4529-b611-bc8d1d7650ed None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.543931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f63b0-542c-4e58-984f-830575f77be6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.552202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.552875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.553160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.553434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.554563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:36.554870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:36.556574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a073e-4987-45dd-8a2d-5582821785ae None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:36.566068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 153/301] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:36.567730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.568019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.568297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.574364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.585733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:36.586110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.586952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.587232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.587504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.588097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:36.600475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:36.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:36.609856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:36.610686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 149/302] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:36.612367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.674954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 154/303] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:36.678667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.742220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e415232-4327-4ff7-b77d-3e2711c70bd9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.781216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.788179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.789065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.789519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.789689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] PATH_INFO: `/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.790911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] RBAC: Authorizing `identity:get_user(user_id=4b6c7c9fc9d44dea9ad32b5395239bad)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:36.792140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cedca-b921-42a4-9acd-8d622cdfab54 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.792374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:36.799055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 150/304] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:36.799055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.834920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.835583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.835861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.836130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.836860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:36.837160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:36.838560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:36.840089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:36.848582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 155/305] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:36.849428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.914413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d91bc-6108-44c1-bc26-a5c1348f458c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:36.951934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:36.952670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:36.952950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:36.953222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:36.954841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:36.956897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:36.962214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 151/306] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:36.962214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:36.970281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401516a-884e-4653-9182-eb4ed867ca53 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.013739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.014884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.015225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.015518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] PATH_INFO: `/v3/system/users/4b6c7c9fc9d44dea9ad32b5395239bad/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.017862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4b6c7c9fc9d44dea9ad32b5395239bad, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:37.019809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:37.031108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 156/307] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jun 30 17:39:36 2022] PUT /identity/v3/system/users/4b6c7c9fc9d44dea9ad32b5395239bad/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:37.032102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:37.040333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:37.087464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697eab51-fbea-4a9e-8bbe-12012b02b8c8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.138946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:37.141573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.142614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.142901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.143187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] PATH_INFO: `/v3/users/348734dba89249898981d7848a9e340c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.173310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] RBAC: Authorizing `identity:get_user(user_id=348734dba89249898981d7848a9e340c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:37.176804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:37.180435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 152/308] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:36 2022] GET /identity/v3/users/348734dba89249898981d7848a9e340c => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:37.187327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:37.206141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe2e51-023f-4b56-86e0-30e1b2a2f821 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.256371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.257050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.257332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.257608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] PATH_INFO: `/v3/groups` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.258304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:37.260656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308ae93-7eef-49b0-89e3-bc29be6f3963 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:37.283022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 157/309] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jun 30 17:39:37 2022] POST /identity/v3/groups => generated 229 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:37.320159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a21d-cf7f-4c02-aa4b-af77dfb2a75e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.349406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:37.360838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.361750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.362039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.362441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.363645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, user.id=348734dba89249898981d7848a9e340c, scope.system=all) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:37.363961 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:37.365436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333870b4-1cc4-4713-886a-411a0022ae2f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:37.372516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 153/310] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 17:39:37 2022] GET /identity/v3/role_assignments?user.id=348734dba89249898981d7848a9e340c&scope.system=all&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:37.393703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ded4a-ab96-410e-b7c8-d521a8fe8584 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.394051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ded4a-ab96-410e-b7c8-d521a8fe8584 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.394273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ded4a-ab96-410e-b7c8-d521a8fe8584 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.396028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 158/311] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:37.402078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.402466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.402791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.425433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:37.427721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:37.528527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 154/312] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:37.552878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ded4a-ab96-410e-b7c8-d521a8fe8584 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:37.662485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ded4a-ab96-410e-b7c8-d521a8fe8584 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.684230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:37.701471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.702297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.702656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.702936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.713996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:37.722405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:37.724752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:37.726745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 159/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:37.731437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:37.841309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569518-5a9b-4681-9d7a-bdda4622c1a3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:37.862239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:37.880474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:37.881150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:37.881435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:37.881711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:37.882498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:37.882809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:37.884203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:37.893238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 155/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:37.898698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:38.010536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5aee36-765f-41ab-91fc-6840cd2b1da0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:38.045756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:38.046659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:38.046946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:38.047223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] PATH_INFO: `/v3/users/b254f1f98a93462b91607aea8732f76b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:38.048422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] RBAC: Authorizing `identity:get_user(user_id=b254f1f98a93462b91607aea8732f76b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:38.049880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:38.051905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 160/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:37 2022] GET /identity/v3/users/b254f1f98a93462b91607aea8732f76b => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:38.056942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:38.164788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1ab2c-f280-4580-ba5b-fc07b1bde641 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:38.199231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:38.199916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:38.200192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:38.200472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:38.250145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:38.271098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:38.272565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:38.376248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 156/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:38.381724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:38.432689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:38.488748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312135e-ccc6-4641-850f-f72a1f0446f1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:38.527794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:38.528677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:38.528958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:38.529236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:38.530580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:38.532113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27e7eb-99d5-4a3b-bc86-a6e5e8ff7aa5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:38.533884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 161/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:38 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:38.539796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:38.625840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:38.645392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3493fb-37e0-4133-b1e9-3ef66bc095c5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:38.680062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:38.680716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:38.681000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:38.681280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:38.682105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, scope.project.id=4823bf889e7d449c85d03bbbfc97c530, user.id=b254f1f98a93462b91607aea8732f76b) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:38.682499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:38.684137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51b9cc-4c39-43d9-9ace-346291913682 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:38.691003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 157/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:39:38 2022] GET /identity/v3/role_assignments?user.id=b254f1f98a93462b91607aea8732f76b&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:39.275800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d8a91-1d1f-44c8-8d72-7cca36dc1e11 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.276301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d8a91-1d1f-44c8-8d72-7cca36dc1e11 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.276301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d8a91-1d1f-44c8-8d72-7cca36dc1e11 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.277788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 162/319] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:39.283321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1d56-0f36-4021-b686-3e0662a54e9c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.283731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1d56-0f36-4021-b686-3e0662a54e9c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.283935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1d56-0f36-4021-b686-3e0662a54e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.301131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-be3b1d56-0f36-4021-b686-3e0662a54e9c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:39.302319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-be3b1d56-0f36-4021-b686-3e0662a54e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:39.369004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 158/320] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:39.392261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d8a91-1d1f-44c8-8d72-7cca36dc1e11 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.453389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976b967-e068-443c-a80c-760c6569d57b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.453695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976b967-e068-443c-a80c-760c6569d57b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.453956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976b967-e068-443c-a80c-760c6569d57b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.455515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 159/321] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:39.460180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e8fa5-64bf-4c5c-b825-7b3cebe30782 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.460451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e8fa5-64bf-4c5c-b825-7b3cebe30782 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.460735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e8fa5-64bf-4c5c-b825-7b3cebe30782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.477581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b95e8fa5-64bf-4c5c-b825-7b3cebe30782 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:39.478810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b95e8fa5-64bf-4c5c-b825-7b3cebe30782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:39.496146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d8a91-1d1f-44c8-8d72-7cca36dc1e11 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:39.512008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:39.531022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:39.531745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.532029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.532307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.542532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:39.545768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 160/322] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:39.547161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d783e7a-3b09-4fc7-ab85-fb562c7dc915 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.547451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d783e7a-3b09-4fc7-ab85-fb562c7dc915 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.547724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d783e7a-3b09-4fc7-ab85-fb562c7dc915 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.549144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 161/323] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:39.550349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:39.551192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:39.551986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 163/324] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:39.557134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.557254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.557422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.557701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.563405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:39.574862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:39.576057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:39.643244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 162/325] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:39.644066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.668288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb635-01f4-4cdb-8d1e-928f8e1f15d4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:39.701765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:39.702967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.703253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.703530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.704298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:39.704605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:39.705971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:39.714029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 164/326] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:39.714835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.743713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92430a-b3c5-450c-bcca-f55fb48c034b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:39.753399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:39.776917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:39.777799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.778084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.778380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.779544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:39.781634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:39.783229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 163/327] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:39.783988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.786546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:39.818355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fa411-ecf5-43df-8b29-1023a9ac2b0c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:39.852216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:39.853005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.853288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.853561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.863510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:39.871200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:39.872125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 165/328] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:39.873140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.886586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351db418-f7f8-4b12-b869-449feed0ec08 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:39.920687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:39.921571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:39.921852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:39.922126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] PATH_INFO: `/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:39.923342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b6c7c9fc9d44dea9ad32b5395239bad)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:39.924781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:39.926428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 164/329] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/users/4b6c7c9fc9d44dea9ad32b5395239bad => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:39.927188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:39.969299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:39.972815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a679-c8a5-4cc2-8bd9-151b0abe9b0c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:40.000628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:40.007024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:40.007720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:40.008000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:40.008276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] PATH_INFO: `/v3/groups` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:40.008961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:40.010460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:40.029156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 166/330] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jun 30 17:39:39 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:40.030128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:40.033913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e96dab-7749-4b01-86bb-8211c5dfc4f7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:40.079620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:40.080675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:40.081112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:40.081544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:40.082520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:40.082988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:40.085269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:40.099728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 165/331] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:40.101285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:40.158979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5842a-9d13-4f1f-9d1e-faabc8f05956 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:40.200934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:40.201602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:40.201913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:40.202192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:40.203061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] RBAC: Adding query filter params (role.id=0e71d3efba6d41a4a468c42031d0320e, user.id=4b6c7c9fc9d44dea9ad32b5395239bad, scope.system=all) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:40.203378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:40.204831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:40.216915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 167/332] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 17:39:40 2022] GET /identity/v3/role_assignments?user.id=4b6c7c9fc9d44dea9ad32b5395239bad&scope.system=all&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:40.225500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f33b5-f106-4f12-a894-3642b450a1c5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:40.276716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:40.284394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:40.285232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:40.285518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:40.285794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] PATH_INFO: `/v3/users/348734dba89249898981d7848a9e340c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:40.287211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] RBAC: Authorizing `identity:get_user(user_id=348734dba89249898981d7848a9e340c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:40.288767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b64e-3056-4556-af1f-ff9ce66d2d03 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:40.291048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 166/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:40 2022] GET /identity/v3/users/348734dba89249898981d7848a9e340c => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:40.296738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:40.319341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:40.418931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8df784-7933-4d63-9252-ed14c80b580b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:40.457589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:40.458492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:40.458780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:40.459057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] PATH_INFO: `/v3/system/users/348734dba89249898981d7848a9e340c/roles/45bbda86ddc84212b101d67cae3f62ca` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:40.460714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=348734dba89249898981d7848a9e340c, role_id=45bbda86ddc84212b101d67cae3f62ca)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:40.462111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabdcc4-5978-42b5-9b9b-3bc0affc3f1b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:40.471493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 168/334] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jun 30 17:39:40 2022] PUT /identity/v3/system/users/348734dba89249898981d7848a9e340c/roles/45bbda86ddc84212b101d67cae3f62ca => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:39:42.520779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f168aef-984a-4703-9a22-287711230695 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:42.521281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f168aef-984a-4703-9a22-287711230695 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:42.521683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f168aef-984a-4703-9a22-287711230695 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:42.523831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 167/335] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:42.540795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:42.541286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:42.541605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:42.566657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:42.568135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:42.691524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 169/336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:42.735279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f168aef-984a-4703-9a22-287711230695 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:42.877604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f168aef-984a-4703-9a22-287711230695 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:42.926463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:42.927419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:42.927790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:42.928127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:42.951362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:42.961695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:42.962653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:42.963507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:42.964752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 168/337] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:42.969293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:42.993840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:43.129084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42d15d-5758-4e9b-8f7e-138d48f73a7e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:43.182173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:43.182990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.183461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.183798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.184684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:43.184997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:43.187004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5eca8f-ea32-4367-b07e-cb16986065b8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:43.198002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 170/338] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:43.204454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:43.303653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf722-e583-43f1-bf9d-014b5d19d35f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.303997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf722-e583-43f1-bf9d-014b5d19d35f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.304360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cf722-e583-43f1-bf9d-014b5d19d35f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.306410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 171/339] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:43.311915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8fef-ce30-43c1-a232-d69a2910030d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.312211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8fef-ce30-43c1-a232-d69a2910030d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.312486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8fef-ce30-43c1-a232-d69a2910030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.322160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:43.347373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-678e8fef-ce30-43c1-a232-d69a2910030d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:43.348652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-678e8fef-ce30-43c1-a232-d69a2910030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:43.399070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9fa5b-2538-426b-aa33-1c4c04a157f0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:43.481120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:43.481785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.482066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.482373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.503801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 172/340] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:43.505253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9456b-f4d2-4482-9d8b-3cfc3c6e0536 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.505543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9456b-f4d2-4482-9d8b-3cfc3c6e0536 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.505821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9456b-f4d2-4482-9d8b-3cfc3c6e0536 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.508928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 173/341] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:43.514905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.515193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.515476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.525360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:43.552056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:43.553523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:43.654629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:43.656729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:43.679469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 174/342] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:43.680433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:43.769760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:43.806602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 169/343] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:43.807535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:43.859772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ce67c-2733-43cb-b221-23002cfc5f45 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:43.897903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:43.909522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:43.910409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.910799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.911198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.915297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b81ea-6789-4505-9b8f-84cca3f53878 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:43.924962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-2954b8bc-6a51-4173-b702-e650ea94002b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:43.935219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:43.936198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:43.936996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 175/344] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:43.938401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f348-3bf8-404a-be1c-947f9f2acf0e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.938763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f348-3bf8-404a-be1c-947f9f2acf0e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.939113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f348-3bf8-404a-be1c-947f9f2acf0e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.944976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 176/345] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:43.944976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90f348-3bf8-404a-be1c-947f9f2acf0e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:43.961507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:43.962643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:43.963101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:43.963544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:43.974977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:43.984491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:43.985423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:43.986228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:43.989052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 170/346] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:43.990082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.026817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.092262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90f348-3bf8-404a-be1c-947f9f2acf0e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.100808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e34c86-65dd-4f8c-acf8-07d35a6a149f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.130840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.131923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.132210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.132484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.134454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.135836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a8c81-f51e-491c-9763-2b253cbe1a19 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.137587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 177/347] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:43 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:44.139008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a210-00b9-4e07-b94a-c6af4ea33d5e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.140153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a210-00b9-4e07-b94a-c6af4ea33d5e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.140153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a210-00b9-4e07-b94a-c6af4ea33d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.144013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.145142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.146876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.147547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.148340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:44.148706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.150119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.158048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-30e6a210-00b9-4e07-b94a-c6af4ea33d5e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:44.159469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-30e6a210-00b9-4e07-b94a-c6af4ea33d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:44.163674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 171/348] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:44.164517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.238159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.251636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 178/349] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:44.252699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d552-bf7f-4f44-b52f-0145795e738e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.252699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d552-bf7f-4f44-b52f-0145795e738e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.252699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d552-bf7f-4f44-b52f-0145795e738e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.260748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 179/350] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:44.260748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d552-bf7f-4f44-b52f-0145795e738e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.273081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.301298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112d4b8-91aa-4d20-9d6f-706d6cf30688 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.346675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.347346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.347629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.347906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.348648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] RBAC: Adding query filter params (name=member) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:44.348953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.350349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.360835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 172/351] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:44.361664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.388958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d552-bf7f-4f44-b52f-0145795e738e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.433886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.435082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.435539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.435978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.442908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.444587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d04ad9-38e8-47dd-bb85-03dd08cba7ef None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.446717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 180/352] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:44.448368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.448822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.449261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.469039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:44.470802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:44.496876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67426ee-6834-4841-a926-16bdabc1d0ba None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.535560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.548212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.549005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.549298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.549576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.562126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 181/353] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:44.563191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.568236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.630294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.631795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.694237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af150d65-81dc-4b0f-8b04-4a34d8d5f9ad None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.744791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.745602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.745887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.746159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.761891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:44.771431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:44.772371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:44.773165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 182/354] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:44.773848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.776228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 173/355] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:44.777620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:44.825359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.826496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:44.891097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36941ac4-3c32-486d-b461-f519f1137b13 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.893441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8c91-704f-4f09-a566-dc5fa4a002bc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:44.931483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.932140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.932431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.932720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.933647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:44.933970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:44.935443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:44.935584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:44.936747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:44.937048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:44.937323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:44.941980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 174/356] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:44.943083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.048335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.049957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.097554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b516c2-00db-4113-9713-e19e6ca4ff81 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.106816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.113145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.139344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.140923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.142100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.142605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.160312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-56fd70d6-2b49-4697-8004-170e3513021d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:45.168269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:45.169379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:45.170501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 175/357] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:45.172212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b2bd-a96d-4cbd-8571-b2c0305a74b5 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.172712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b2bd-a96d-4cbd-8571-b2c0305a74b5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.173191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05b2bd-a96d-4cbd-8571-b2c0305a74b5 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.175224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 176/358] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:45.180532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05b2bd-a96d-4cbd-8571-b2c0305a74b5 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.198586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 183/359] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:45.199770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.313615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05b2bd-a96d-4cbd-8571-b2c0305a74b5 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.336324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7cbe-446c-45c7-a723-c9a2eb39f686 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.353741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.354933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.355396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.355978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.357197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:45.357657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.371834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 177/360] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.372956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.383985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.384269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.384563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.384843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.386067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.387667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebde6f-a341-4c24-b749-31ebf2b03ea2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.389548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 184/361] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.390834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.395187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.395187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.412765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:45.413990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:45.418626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.425366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.520029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 185/362] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:45.520704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.526405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ef1c9-b796-4b51-ace5-3dc1f7415a21 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.581499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.581499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.581499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.581499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.581883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:45.581883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.583286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.598429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 178/363] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.598975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.651500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448bba5-781d-4edb-bf35-3e7befab2f22 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.697102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.697969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.698567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.698847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.701399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.702961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.704893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 186/364] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.705495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.725179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.727814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:45.734583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cc910-0dee-44dc-b7f2-a4cc68a4b5a2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.785713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.786892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.787174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.787444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] PATH_INFO: `/v3/users/348734dba89249898981d7848a9e340c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.790215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] RBAC: Authorizing `identity:get_user(user_id=348734dba89249898981d7848a9e340c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.791709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.794012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 179/365] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/users/348734dba89249898981d7848a9e340c => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.794866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.834455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632526d-cec5-4adc-a497-07e730f5b780 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.874459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.875318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.875599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.875872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.877045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:45.878520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:45.880931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 187/366] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:45.881053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.929657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd390-eb87-43cd-803d-3721c4e64b76 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:45.967845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:45.968805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:45.969088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:45.969414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:45.982559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:45.988759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:45.989936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 180/367] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:45.991024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:45.997942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.022733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a008-ec9f-48d0-b1cd-779d558a22b5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.027818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.067231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.068151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.069115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.069546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.129316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e7b7-b1f5-49b5-8b88-3076527bab55 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.152553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.154259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.163475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.164452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.164745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.165023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.166315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.167787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.174407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 181/368] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.174407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.235644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.264855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.265041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 188/369] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:46.265968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.279367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f8fc4-17fe-4600-9343-d58a4d6c6354 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.314068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.314750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.315045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.315326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.316135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] RBAC: Adding query filter params (role.id=45bbda86ddc84212b101d67cae3f62ca, user.id=348734dba89249898981d7848a9e340c, scope.system=all) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:46.316448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.317870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.330717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 182/370] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/role_assignments?user.id=348734dba89249898981d7848a9e340c&scope.system=all&role.id=45bbda86ddc84212b101d67cae3f62ca => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.339377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.379919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8a01a-b251-466c-927c-b015f64eec8d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.421295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.422161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.422592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.423068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.424140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:46.424552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.427114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.439219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.439219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.510104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.522998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1a10b-a83d-4578-9880-e24b53e1ac20 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.566880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.567612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.567863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.568137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.568899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:46.569210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.570652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.582308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 183/372] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.583043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.590886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938d21-dc26-463f-be3b-a1cfce11ceed None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.601313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.645935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.646684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.646972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.647290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.648245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=4823bf889e7d449c85d03bbbfc97c530) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:46.648949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.650694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.660465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.661452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.727261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377f55c-0557-47cf-bbb4-23ac4730fc1a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.774280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.775435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.775719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.776514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.778075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.780769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.782859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:46.788998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:46.804549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41828bfa-9aa7-4e6b-9bc9-ec62b5f60011 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.813195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.847637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.848416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.848705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.849086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.878843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:46.904154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aaa1e-e574-48aa-8a5a-899772e4788b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:46.929722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.931283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.943820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:46.944686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:46.944967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:46.945242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:46.946491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:46.947955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:46.950221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 185/375] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:46.959867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.091969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 191/376] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:47.096559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7283-feaa-4196-b4cd-86904f406e16 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.098856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.113585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:47.156661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.158501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.158792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.159079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.160742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=4823bf889e7d449c85d03bbbfc97c530) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:47.161499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:47.163999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20b851-5481-4382-bea2-2b25a95f0f80 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:47.173013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 186/377] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:46 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:47.174509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59b23-a75f-4ad1-874c-2c0d618f0251 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.174804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59b23-a75f-4ad1-874c-2c0d618f0251 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.175083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd59b23-a75f-4ad1-874c-2c0d618f0251 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.176423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 187/378] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:47.182870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.183168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.183437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.205604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:47.207371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:47.210120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:47.293531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 188/379] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:39:47.309002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bde92-0092-4974-bced-7bcce3f4fab0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.319576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.388551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.389977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.390254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.390576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] PATH_INFO: `/v3/projects/2b7acc57e2754d7ab425d395278a2937` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.393264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] RBAC: Authorizing `identity:get_project(project_id=2b7acc57e2754d7ab425d395278a2937)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:47.395006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:47.401449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 192/380] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/projects/2b7acc57e2754d7ab425d395278a2937 => generated 344 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:47.402444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.448553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c078e-c1c5-4936-bffe-d306ced5c13c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.506576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.507397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.507678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.507957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.522951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:47.525346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:47.533293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:47.548455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:47.551099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:47.552101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 189/381] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:47.558367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.569112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d9101-a233-48e5-9fdd-7e01120d5f21 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.635459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.636331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.636618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.636897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] PATH_INFO: `/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.649743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] RBAC: Authorizing `identity:get_group(group_id=6bd9ecbf3d5f42718c594a6787b9a34f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:47.651191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:47.653198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 193/382] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:47.659246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.772757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538831bd-85f1-48c1-b273-0a8b4e1a41ae None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.781228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63921-778b-4524-bba5-c78a78f2ce89 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:47.809538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.810209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.810524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.810813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.812223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] RBAC: Adding query filter params (group.id=6bd9ecbf3d5f42718c594a6787b9a34f, role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=2b7acc57e2754d7ab425d395278a2937) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:47.812539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:47.813974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:47.820206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 194/383] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/role_assignments?group.id=6bd9ecbf3d5f42718c594a6787b9a34f&scope.project.id=2b7acc57e2754d7ab425d395278a2937&role.id=1df4096f023c48ccab33e5939498b3da => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:47.853812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:47.855234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:47.855704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:47.856163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:47.857124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:47.857618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:47.859973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:47.877439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 190/384] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:47.883131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:47.888690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:48.031914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2acbce-fcfc-4d7c-975c-913fd74527af None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:48.074001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:48.074896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:48.075185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:48.075466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:48.079038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:48.080487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:48.082133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 195/385] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:47 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:48.087745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:48.174525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:48.223970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dae11-6455-4ee4-a35e-64564f5256a8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:48.259281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:48.260514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:48.260996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:48.261468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:48.262890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:48.265209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7aa235-bfc8-4cf7-bb79-74ad9697e85b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:48.267480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 191/386] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:48 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:48.274356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:48.390197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:48.426344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b639bc-ea32-462e-9f6d-d40129b6fa5f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:48.476899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:48.477787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:48.478076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:48.478369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:48.480389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, group_id=b5cb25068a4f4fdb8a0d7c841884ff64, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:48.481846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452da76-c8c3-457e-9f31-b4e46981b99a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:48.512905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 196/387] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 17:39:48 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:49.085696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659b7f-cd3b-40cd-a97d-3311b57219c0 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.086716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659b7f-cd3b-40cd-a97d-3311b57219c0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.087212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59659b7f-cd3b-40cd-a97d-3311b57219c0 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.088896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 192/388] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:49.094944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974ca7-fcd5-492c-bd89-91eaebcfa0ec None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.095413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974ca7-fcd5-492c-bd89-91eaebcfa0ec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.095603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974ca7-fcd5-492c-bd89-91eaebcfa0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.124374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-31974ca7-fcd5-492c-bd89-91eaebcfa0ec None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:49.125679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-31974ca7-fcd5-492c-bd89-91eaebcfa0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:49.214167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 197/389] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:49.244907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59659b7f-cd3b-40cd-a97d-3311b57219c0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:49.257931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc126-b123-410f-9b1a-3ec3eb4f60b4 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.258228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc126-b123-410f-9b1a-3ec3eb4f60b4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.258535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dc126-b123-410f-9b1a-3ec3eb4f60b4 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.260297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 198/390] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:49.266057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.266384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.266774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.294657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:49.295892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:49.399084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59659b7f-cd3b-40cd-a97d-3311b57219c0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:49.408687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:49.427792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 199/391] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:49.438677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:49.454395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:49.455211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.455495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.455757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.468804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:49.486642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:49.502664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:49.503564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:49.504373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 193/392] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:49.509686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:49.672368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1b25e-f113-4670-bddf-82526c1f3fa2 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:49.694028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd62def-d2fa-4765-aa91-099b4b08c17e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:49.744449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:49.745120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.745402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.745676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.746717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:49.747032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:49.749184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:49.758236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 194/393] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:49.763066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:49.775060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:49.776072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.776501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.776916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.796477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f6241e16-28ff-4711-8262-b9fca8adfe55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:49.804961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:49.805899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:49.806749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:49.814762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 200/394] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:49.816431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679a0-d69f-4b79-8655-d11b1c429baa None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.817212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:49.817681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679a0-d69f-4b79-8655-d11b1c429baa None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.818171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31679a0-d69f-4b79-8655-d11b1c429baa None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.819941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 201/395] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:49.820885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31679a0-d69f-4b79-8655-d11b1c429baa None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:49.882525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:49.883584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085089-bfe0-4245-9194-59d3e7d91dfe None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:49.919708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:49.920922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.921380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.921828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.923265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:49.924890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ca73-fafc-4520-aec4-da3f217e61d3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:49.927087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 195/396] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:49 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:49.928532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.929020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.929494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.947068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:49.947456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31679a0-d69f-4b79-8655-d11b1c429baa None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:49.949077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:49.991072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:49.991910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:49.992253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:49.993085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:49.993955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:49.994326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:49.996167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.012788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 202/397] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.012788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.029288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 196/398] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:50.035803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.109669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.141294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f5b3c-6e8b-4c8b-bb87-f2487a965699 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.159525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.171253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dacf32-a5c8-43e1-b4f7-a8eee1402e91 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.196437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.197302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.197584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.197881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.199538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.201000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.206664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 203/399] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.207470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.213990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.215242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.215733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.216198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.218168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.220284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.222826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 197/400] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.224024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.318868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6f5e4-b079-4e7b-9576-508bd26ac867 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.336301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b52df-25e5-404c-a170-7e4d5425c786 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.353608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.354015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.354861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.355414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.355720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.367088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:50.372375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.373288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.374075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.374369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:50.375823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:50.376656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 204/401] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:50.382448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.383174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3daf7c96eeb40c2900e76a782944d4c, group_id=b5cb25068a4f4fdb8a0d7c841884ff64, role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.384291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e75192-c74c-4409-b66e-48fc4f9fdca7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.398125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 198/402] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 17:39:50 2022] PUT /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:50.399748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ae89-5fbc-4ec5-ad71-238c4b601d2d None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.400107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ae89-5fbc-4ec5-ad71-238c4b601d2d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.400450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ae89-5fbc-4ec5-ad71-238c4b601d2d None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.401975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 199/403] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:50.402670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ae89-5fbc-4ec5-ad71-238c4b601d2d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.433366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.504419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab3b85-d4e4-4de2-aca1-951517ee77b4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.543727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.544751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.545686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.546008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.547480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.549619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc1bc2-db69-4dc0-b5e9-4b6abf6d9e38 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.551573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 205/404] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.557701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.558063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.558435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.567040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ae89-5fbc-4ec5-ad71-238c4b601d2d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.579359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:50.580498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:50.614803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.615481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.615761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.616040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.616794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:50.617108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.619407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.628620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 200/405] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.633250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.661089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 206/406] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:50.670722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.683319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.725752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.775265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f72a-9194-4553-979f-128bead0987f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.802136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f129dd-a6cc-4cb4-8968-52a1ad8ffa11 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:50.812729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.813555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.813839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.814114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.816312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, group_id=b5cb25068a4f4fdb8a0d7c841884ff64, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.817926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.838483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 201/407] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 17:39:50 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:50.838483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.854886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:50.856245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:50.856388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:50.856742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:50.863319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:50.865317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:50.867470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 207/408] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:50.873358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:50.953883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:50.970893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f45ad-21bc-4c36-a6f5-06b28ed8fc0c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.054142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.054987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.055272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.055550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.074176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5feb98-1306-44c0-aae5-d259598db89e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.074920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:51.083697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-7932d30e-4828-438d-904c-ac46a3eea64f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:51.084727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:51.085585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:51.086480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:51.087979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 202/409] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:51.089251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe492b-0911-43ae-9c08-de0bc51dd08d None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.089536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe492b-0911-43ae-9c08-de0bc51dd08d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.089826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe492b-0911-43ae-9c08-de0bc51dd08d None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.091268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 203/410] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:51.095153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe492b-0911-43ae-9c08-de0bc51dd08d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.120835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.123477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.123477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.123672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.125882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.128006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df4821-a859-4940-b453-46ff820f7fac None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.129991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 208/411] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:50 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:51.137267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.137555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.137835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.158713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:51.160062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:51.218122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe492b-0911-43ae-9c08-de0bc51dd08d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.238668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 209/412] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:51.239567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.264499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.265455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.266209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.267124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.268338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:51.269284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.271819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.283850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 204/413] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:51.289348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.317644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:51.358469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1458cd-986e-4f78-b933-4dc101ce3bb8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.361363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:51.405334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.406914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.407637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.408551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.411570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] RBAC: Authorizing `identity:create_grant(project_id=4823bf889e7d449c85d03bbbfc97c530, group_id=b5cb25068a4f4fdb8a0d7c841884ff64, role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.413322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.414087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebcfd-c1b2-460e-a551-7e20291493fa None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.434434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 210/414] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 17:39:51 2022] PUT /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530/groups/b5cb25068a4f4fdb8a0d7c841884ff64/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:39:51.435466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.477114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.478078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.478381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.478662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] PATH_INFO: `/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.480820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] RBAC: Authorizing `identity:get_group(group_id=6bd9ecbf3d5f42718c594a6787b9a34f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.483731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.485860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 205/415] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:51.492403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.590221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a57de2-94f1-4239-8e89-77f50fd691bd None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.672261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.672983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.673263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.673535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.686582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:51.686771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:51.695422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:51.696391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:51.697632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 211/416] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:51.701985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07635f5-6c26-45df-b9ea-69b1d3a8b10e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.707630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.750629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:51.755773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.759388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.760027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.760470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] PATH_INFO: `/v3/projects/2b7acc57e2754d7ab425d395278a2937` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.762219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] RBAC: Authorizing `identity:get_project(project_id=2b7acc57e2754d7ab425d395278a2937)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.765953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.768454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 206/417] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/projects/2b7acc57e2754d7ab425d395278a2937 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:51.774242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.838078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7a113-4aba-4303-97a4-5127b1d278d2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.875688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.876368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.876668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.876943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.877697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:51.878009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.879444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.883859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1a65-65d0-4794-889d-34414aa541ce None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:51.888394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 212/418] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:51.894576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:51.921205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:51.922511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:51.922996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:51.923456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] PATH_INFO: `/v3/projects/2b7acc57e2754d7ab425d395278a2937/groups/6bd9ecbf3d5f42718c594a6787b9a34f/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:51.925628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b7acc57e2754d7ab425d395278a2937, group_id=6bd9ecbf3d5f42718c594a6787b9a34f, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:51.927291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:51.954374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 207/419] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jun 30 17:39:51 2022] PUT /identity/v3/projects/2b7acc57e2754d7ab425d395278a2937/groups/6bd9ecbf3d5f42718c594a6787b9a34f/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:39:51.990778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:52.007958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b9801-fefd-481f-bba3-2e520b7b1eeb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:52.046937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:52.047597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.047884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.048164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.129478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:52.131006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:52.226109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:52.247910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 213/420] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:52.251466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:52.272323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:52.376975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebbebd-bb74-445f-97d9-0e9785966789 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:52.436535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:52.437852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.438394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.438928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.440432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:52.442075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:52.444081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 208/421] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:52 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:52.449889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:52.581902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b619e-9c80-42a4-985d-839f49bdb6d9 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:52.617684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:52.618962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.619469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.619981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.621365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:52.623090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b1ad4-9478-47c9-bf52-8532265c7824 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:52.625303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 214/422] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:52 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:52.630663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:52.735883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:52.752754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954505f-83ee-4e10-b123-a951fae896e8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:52.770822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab3bd-382a-4bb1-9fd7-fd1726d5d08e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.771121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab3bd-382a-4bb1-9fd7-fd1726d5d08e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.771389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcab3bd-382a-4bb1-9fd7-fd1726d5d08e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.773046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 215/423] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:52.787769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.788056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.788330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.808496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:52.809716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:52.815803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:52.816734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:52.817286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:52.817783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:52.818859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:52.819412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:52.821067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:52.833892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 209/424] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:52 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:52.871019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:52.888030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 216/425] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:52.927784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.082178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83c3a-acf8-405c-add5-fec7d24b0210 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.117325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.118220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.118535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.118817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.131861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:53.140907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:53.141815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:53.142740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 210/426] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:53.148295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.280383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e350b0-6c89-42c2-85a4-f27c9a5e1d4a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.316502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.317174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.317457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.317728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.318578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:53.318884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:53.320314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a016c16-f946-4f99-ba6e-a74828418313 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:53.330265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 217/427] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:53.335631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.351872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:53.464248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599dad9-6309-45fe-ac81-646d853bffa8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.470395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ea2-3ca6-4680-a933-1285e943997f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.470696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ea2-3ca6-4680-a933-1285e943997f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.470957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae448ea2-3ca6-4680-a933-1285e943997f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.472696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 218/428] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:53.480646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.480952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.481212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.508686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:53.510047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:53.510184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.510806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.511090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.511369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.570091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:53.595451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:53.596951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:53.602522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 219/429] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:53.608725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:53.628398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.730659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 211/430] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:53.736020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.757800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b590-fad2-46d4-baeb-96e00b3189d7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.794628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.795973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.796250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.796517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.809080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:53.817531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:53.818537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:53.819359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 220/431] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:53.828405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.847306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d840b-c017-401d-8439-924e7d1e01b2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.859405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:53.871120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:53.889847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.890891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.891272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.891666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] PATH_INFO: `/v3/projects/c3daf7c96eeb40c2900e76a782944d4c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.893070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=c3daf7c96eeb40c2900e76a782944d4c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:53.894638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:53.903431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 212/432] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/projects/c3daf7c96eeb40c2900e76a782944d4c => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:53.905701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:53.945928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b3ff-8938-4b80-ace0-f5136b87274d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:53.985308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:53.985900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:53.986185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:53.986493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:53.987588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] RBAC: Adding query filter params (name=member) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:53.987908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:53.990137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.006110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 221/433] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:54.007551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.033717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6cdb6-c666-49f5-a75c-ba0204d5b9f2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.073775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.075332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.075995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.076449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.078261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.080506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.083480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 213/434] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:53 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:54.088794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.130055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.134022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ef328-b60e-4bd5-a589-d875db43616f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.141802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.175022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.176228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.176532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.176811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.202586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f57585-74a9-4043-9550-2d05642a55a8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.239524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.240089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.240370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.240643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.241469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=c3daf7c96eeb40c2900e76a782944d4c) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:54.241774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.243244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193500de-4442-491c-a145-970674b2d5d4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.249885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 214/435] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=c3daf7c96eeb40c2900e76a782944d4c&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:54.251849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea34ef-5a7f-41c2-9bba-cec86ab66966 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.252328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea34ef-5a7f-41c2-9bba-cec86ab66966 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.253068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea34ef-5a7f-41c2-9bba-cec86ab66966 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.255848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 215/436] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:54.257233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.262709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.265074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.265074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.265074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.285090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:54.286410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:54.371132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 216/437] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:54.381831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.388021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 222/438] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:54.392418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.394049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.444827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.516779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7390d1-e31a-4916-bbdf-4eb0e79b4395 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.531181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e424d-5829-4e15-8aa1-5ea4a3d0e835 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.566113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.567958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.568241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.569655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.571116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.572587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.573931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.574933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 223/439] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:54.575245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.575735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.576747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.580976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.593558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:54.603407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:54.604373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:54.605251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 217/440] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:54.611397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.674281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.721148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fcf8f-fcf8-4d31-b199-6e45441a7013 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.741214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.746213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e956184-dc3b-45a2-a95f-24c52d9632db None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.758278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.759165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.759453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.759739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.760925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.762399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a6968-c673-4bbb-9478-be46c77b59ea None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.764710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 224/441] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:54.765892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941110-1063-4f3e-aba6-911fc7aa9d50 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.766181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941110-1063-4f3e-aba6-911fc7aa9d50 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.766490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6941110-1063-4f3e-aba6-911fc7aa9d50 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.767798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 225/442] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:54.770525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6941110-1063-4f3e-aba6-911fc7aa9d50 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.785870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.786584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.786871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.787153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.787904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:54.788215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.789603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb65bd-7882-44c8-9b7a-9dd2350bcdd5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.797794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 218/443] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:54.798928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.799218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.799497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.815226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:54.816423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:54.876409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6941110-1063-4f3e-aba6-911fc7aa9d50 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:54.882505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 219/444] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:54.883350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.922496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:54.923151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:54.923447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:54.923705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:54.924536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=0e71d3efba6d41a4a468c42031d0320e, scope.project.id=4823bf889e7d449c85d03bbbfc97c530) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:54.924851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:54.926282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:54.928195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.933484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 226/445] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=0e71d3efba6d41a4a468c42031d0320e => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:54.934291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:54.981800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:54.998651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523ce13-08f5-49c6-b846-88f4b557c5b1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.039145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.039725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.040003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.040274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.053233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344f1d-7d80-4c61-8520-acde4ca0346d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.095161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.095946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.096227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.096498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.109364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:39:55.117162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:55.118083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:39:55.118933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:55.119578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 227/446] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:55.124847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.126620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.128091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.210321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.239274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.256009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 220/447] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:39:55.262933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.266402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb760e2-fc82-460b-8546-b1577c8e35ca None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.320106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.320784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.321109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.321386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.322195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:55.322554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.324221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.335019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 228/448] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:55.340944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.383835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9cb04-aa38-4e72-afda-c0c5775e7cc2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.419599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.421006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.421288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.421561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] PATH_INFO: `/v3/projects/4823bf889e7d449c85d03bbbfc97c530` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.422769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] RBAC: Authorizing `identity:get_project(project_id=4823bf889e7d449c85d03bbbfc97c530)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.424223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.430517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 221/449] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/projects/4823bf889e7d449c85d03bbbfc97c530 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:55.431269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.470891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606f41-e962-46e1-a357-e9310897e722 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.491007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.492104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.508284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.508958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.509235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.509505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.540877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1af95c-fb06-4aba-b1b0-84c34f00d73a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.581564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.582364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.582645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.582916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] PATH_INFO: `/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.584086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] RBAC: Authorizing `identity:get_group(group_id=b5cb25068a4f4fdb8a0d7c841884ff64)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.585516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.586713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.587154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 222/450] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/groups/b5cb25068a4f4fdb8a0d7c841884ff64 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:55.588188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.592874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.688101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 229/451] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 17:39:55.693589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.695376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a7528-fda8-4b6b-85de-f83a73de3c30 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.714918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.721970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:55.731001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.731659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.731942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.732218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.733602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] RBAC: Adding query filter params (group.id=b5cb25068a4f4fdb8a0d7c841884ff64, role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=4823bf889e7d449c85d03bbbfc97c530) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:55.733915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.735395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.741538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 223/452] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/role_assignments?group.id=b5cb25068a4f4fdb8a0d7c841884ff64&scope.project.id=4823bf889e7d449c85d03bbbfc97c530&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:55.800832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcf662-8fe0-4c52-a2f8-6c2d9b9d0d87 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.841063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.842428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.842713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.842985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] PATH_INFO: `/v3/projects/2b7acc57e2754d7ab425d395278a2937` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.844252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=2b7acc57e2754d7ab425d395278a2937)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.845723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:55.847561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 230/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/projects/2b7acc57e2754d7ab425d395278a2937 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:55.853198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:55.959893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11db5-6962-42f9-a7eb-e9fce447a67d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:55.995230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:55.996097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:55.996383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:55.996661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] PATH_INFO: `/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:55.997838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] RBAC: Authorizing `identity:get_group(group_id=6bd9ecbf3d5f42718c594a6787b9a34f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:55.999383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1411ac-8a7a-466a-a4a1-590cf68a23b3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:56.001028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 224/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jun 30 17:39:55 2022] GET /identity/v3/groups/6bd9ecbf3d5f42718c594a6787b9a34f => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:56.006945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:56.100966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:56.113720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aa76c-8258-48d0-b4c5-e007299bd7e2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:56.147479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:56.148129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:56.148413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:56.148688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:56.149508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] RBAC: Adding query filter params (group.id=6bd9ecbf3d5f42718c594a6787b9a34f, role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=2b7acc57e2754d7ab425d395278a2937) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:56.149822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:56.151279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133c2c2-1371-46a0-b90f-1c6e9f9faca1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:56.157579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 231/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jun 30 17:39:56 2022] GET /identity/v3/role_assignments?group.id=6bd9ecbf3d5f42718c594a6787b9a34f&scope.project.id=2b7acc57e2754d7ab425d395278a2937&role.id=1df4096f023c48ccab33e5939498b3da => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:58.568718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89d98-18f9-4b5c-a244-1591b2789b68 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:58.569787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89d98-18f9-4b5c-a244-1591b2789b68 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:58.570279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a89d98-18f9-4b5c-a244-1591b2789b68 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:58.572154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 225/456] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:58.578149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ccabb-3cfd-40f7-ab84-f70f1c2e7b1c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:58.579088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ccabb-3cfd-40f7-ab84-f70f1c2e7b1c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:58.579568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ccabb-3cfd-40f7-ab84-f70f1c2e7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:58.602683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-660ccabb-3cfd-40f7-ab84-f70f1c2e7b1c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:58.604282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-660ccabb-3cfd-40f7-ab84-f70f1c2e7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:58.704238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 232/457] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:58.748403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89d98-18f9-4b5c-a244-1591b2789b68 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:58.845610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:58.851837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b293ba-4d5b-476c-ab53-fa2f8073c805 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:58.852423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b293ba-4d5b-476c-ab53-fa2f8073c805 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:58.852907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b293ba-4d5b-476c-ab53-fa2f8073c805 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:58.854491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 233/458] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:39:58.862546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:58.862831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:58.863106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:58.903451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:58.906977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:58.910695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a89d98-18f9-4b5c-a244-1591b2789b68 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:58.959448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:58.960734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:58.961229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:58.961706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:58.984709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:58.988506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:39:59.017915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:59.018925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:59.019793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:39:59.019793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:39:59.019793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.019793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:59.035449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 226/459] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:59.039522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.055569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 234/460] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:59.096302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.126735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.148053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378913e4-fadf-41a4-9d97-9e3e517eb229 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.159809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.184852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.185948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.186485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.186981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.187921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:59.188420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.190032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcce94-d18a-4f8f-9685-cbe73cc0d816 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.204019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 227/461] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:39:59.209938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfe4cb-aab0-4283-98db-ba1a246805ad None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.210696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfe4cb-aab0-4283-98db-ba1a246805ad None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.211193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfe4cb-aab0-4283-98db-ba1a246805ad None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.213045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 228/462] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:59.213959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfe4cb-aab0-4283-98db-ba1a246805ad None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.244905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ab588-000f-4e38-8bac-c2f65681042b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.285534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.286475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.286758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.287031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.300217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.301480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9108787d-5246-4cf8-90a7-059927d0679a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:39:59.322320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:59.323511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:59.324345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:39:59.324345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:39:59.324345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.324345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:59.324345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 235/463] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:39:59.325653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00625-62c3-4d5d-96ae-b09356874f52 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.325928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00625-62c3-4d5d-96ae-b09356874f52 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.326209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c00625-62c3-4d5d-96ae-b09356874f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.329354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfe4cb-aab0-4283-98db-ba1a246805ad None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.345179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c4c00625-62c3-4d5d-96ae-b09356874f52 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:59.346840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c4c00625-62c3-4d5d-96ae-b09356874f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:59.366913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.367944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.368442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.368921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.369817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.371519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.394013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 229/464] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:59 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:39:59.395279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.423921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.430022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.439343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 236/465] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:59.443603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799fe9b-c729-405b-99ff-86a986bd4aec None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.444223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799fe9b-c729-405b-99ff-86a986bd4aec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.444661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799fe9b-c729-405b-99ff-86a986bd4aec None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.446711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 237/466] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:39:59.452510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.454770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.455306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.484672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:39:59.485911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:39:59.539717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcbbfc-18b1-4525-8eb6-bc27c7f650f9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.563477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 238/467] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:39:59.564038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.579898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.581451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.581930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.582437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.583485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:39:59.584117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.585693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.599549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 230/468] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:39:59.601573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.703956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154f36b-fb54-40ae-99a2-a1045f81d230 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.727865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.736700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:39:59.750190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.751030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.751314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.751591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.751924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952d730-3f72-4430-9b30-3f306f8789be None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.763069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.765192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:39:59.785617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:59.786663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:59.787520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:39:59.787520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:39:59.787520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.787520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:39:59.787520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 239/469] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:59.787987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.798495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.799332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.799622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.799906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.816809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.818680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:39:59.844204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:39:59.845135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:39:59.845991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:39:59.845991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:39:59.845991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:39:59.845991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:39:59.848740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 231/470] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:39:59.849847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:39:59.929961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0263bcb-1aee-4c1d-bd26-250935667a98 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.960293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda3fd-b6a1-470c-b4b1-080081fa10a8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:39:59.975824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:39:59.976930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:39:59.977767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:39:59.978209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:39:59.979376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:39:59.981435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.000872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.001728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.002152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.002574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.003436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:00.003874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:00.005366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.008424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 240/471] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:39:59 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:00.009386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:00.027061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:00.030186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 232/472] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:00.039408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:00.067451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:00.160302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b57bb-05df-4a7d-94a2-37b408bfd09b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:00.201681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.202376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.202668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.202943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.203668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:00.203976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:00.206524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.218584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 241/473] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:00.224848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:00.254311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c68241-b86c-4284-8f60-1afadea3549c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:00.295255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.295933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.296217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.296956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.297672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:00.299147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3dcbd-59f5-4bc3-91bb-4eab1f8d6fec None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.331157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 233/474] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:40:00 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:00.332613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51452c-d4c5-4a79-b41f-aeed190fe435 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.332904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51452c-d4c5-4a79-b41f-aeed190fe435 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.333185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51452c-d4c5-4a79-b41f-aeed190fe435 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.334819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 234/475] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:00.341930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.342240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.342520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.352340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:00.365835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:00.367235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:00.371014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78267fb8-6b77-475a-89a3-0f3c09b0d655 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:00.424155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.425254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.425728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.426196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.427220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:00.429720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.440246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:00.466481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 235/476] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:00.471237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 242/477] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:40:00 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:00.504006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:00.640269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f106b-7715-49e0-a2a5-99f97535770f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:00.679539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.680145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.680435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.680717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.681922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:00.682919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:00.693059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.sql.core [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 17:40:00.693059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 17:40:00.693059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [parameters: {'id': '95fab1f4b3684adebdf61f8e49953f92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 17:40:00.693059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:00.710140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:00.711136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application yield resource Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise self.value Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application session.commit() Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 17:40:00.712154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.session.flush() Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application compat.raise_( Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 17:40:00.714036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:00.716521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:00.718953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [parameters: {'id': '95fab1f4b3684adebdf61f8e49953f92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:00.720920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 17:40:00.723550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:00.726834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 236/478] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 17:40:00 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:00.726979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:00.848455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae6430-6f0e-4c01-9191-2115711c669b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:00.889629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:00.890960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:00.891471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:00.891999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:00.904577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c24ab530-257c-4826-acbd-27d30d5a50db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:00.913659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:00.914618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 17:40:00.915467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:00.918399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 243/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:00.926408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:01.016475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:01.097347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a993-721b-4069-9d29-2cb4ab340d80 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:01.143345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:01.144023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:01.144305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:01.144577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:01.145323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:01.145629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:01.147045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0ef24-7ee4-42da-977c-6acc9be008cb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:01.162361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 237/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:01.824502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044656-6d71-48cc-ab0b-ca70443c5447 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:01.825065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044656-6d71-48cc-ab0b-ca70443c5447 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:01.825065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4044656-6d71-48cc-ab0b-ca70443c5447 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:01.826876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 244/481] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:01.832493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:01.832988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:01.833097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:01.851118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:01.852329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:01.940522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 238/482] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:01.993443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4044656-6d71-48cc-ab0b-ca70443c5447 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.009540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:02.136632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4044656-6d71-48cc-ab0b-ca70443c5447 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:02.189507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:02.190309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.190616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.190894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.210512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:02.218947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:02.220244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:02.221043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 245/483] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:02.231493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.302873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:02.378778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86983873-81db-4fca-97e8-6730ecc9b2b1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:02.432980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:02.433576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.433862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.434140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.434921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:02.435757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:02.437150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2739e-d797-4f29-a862-73e4c4774739 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:02.450739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 239/484] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:02.456345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.482359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4a92-16bf-4231-8ead-1eba524b8362 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.482359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4a92-16bf-4231-8ead-1eba524b8362 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.482359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4a92-16bf-4231-8ead-1eba524b8362 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.482359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 240/485] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:02.503039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.503347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.503602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.536391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:02.539732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:02.542590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:02.585886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64015e-a7a1-4721-969e-5535df0afed3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:02.620692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:02.621489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.621773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.622049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.633726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:02.635234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:02.654084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:02.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:02.655974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:02.655974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:02.655974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:02.655974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:02.655974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 246/486] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:02.662513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.682840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 241/487] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:02.729119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.786674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:02.838008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429d88-1e54-4d54-978f-f0990fdaa8b8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:02.866482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:02.887070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:02.890560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.890837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:02.891121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:02.891886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:02.892193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:02.893604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:02.922472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 247/488] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:02.927620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:02.951031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eacd0e-82cb-4738-9d2b-e52f81667831 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:02.998418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:02.999482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:02.999770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.000051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.014630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:03.023586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:03.024569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:03.025386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:03.026005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 242/489] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:03.032599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.155801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.156572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28392081-9e62-46aa-b0d7-1c0bc9b1e392 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.167919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23cd7d-aa5d-459a-a719-28cf501414dd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.198144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.198978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.199258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.199528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.208510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.210041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.212377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.213182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.213577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.213950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.214944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:03.215376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.217407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac205ff0-788c-4e27-b4a4-d47403c697f9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.226608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 243/490] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:03.228099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd87e6-23e1-4f6f-989a-32a84d193abb None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.228486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd87e6-23e1-4f6f-989a-32a84d193abb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:03.228696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:03.229797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:03.230830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:03.231342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd87e6-23e1-4f6f-989a-32a84d193abb None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.231410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 248/491] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:03.234463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 244/492] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:03.235907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.236182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd87e6-23e1-4f6f-989a-32a84d193abb None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.284899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.350633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364dac7a-00bb-4d4d-9d7a-ce42616a36dc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.356006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd87e6-23e1-4f6f-989a-32a84d193abb None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.386967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.387662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.387975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.388260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.389101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:03.389412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.390907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886612d8-72b3-4eef-8248-1d180ee2b9de None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.395637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.396609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.397063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.397482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.408835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.410510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.429420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 249/493] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-ff3c659d-300a-4552-80e6-7eae0484fd32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:03.430520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:03.431536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:03.432593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:03.432593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:03.432593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:03.432593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:03.432832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.432832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.432832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.434552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 245/494] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:03.436020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e0bb-c653-4b30-901a-203564a8221b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.436435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e0bb-c653-4b30-901a-203564a8221b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.436836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e0bb-c653-4b30-901a-203564a8221b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.438491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 246/495] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:03.439191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8e0bb-c653-4b30-901a-203564a8221b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.448478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.449028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:03.450375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:03.532708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 250/496] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:03.533748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.556809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8e0bb-c653-4b30-901a-203564a8221b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.567460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.595989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.596640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.597119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.597443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.649884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19bb7-b7d4-499b-8f20-65442ef608fa None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.668309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.670291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.679308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.686350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.687017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.687308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.687572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.688285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:03.688590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.689983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ce666-abec-442a-9bfb-e0bf7076563a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.702057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 251/497] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:03.703419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.703706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.703978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.721505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:03.723166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:03.787901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 247/498] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:03.788880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.796701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 252/499] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:03.797791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.819561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.906387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd21f2-06d5-41c2-83ad-326feefaddd8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.913821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:03.915916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a772ba-eb80-408c-a3d5-ea800147877f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:03.941291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.942077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.942381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.942659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.953562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:03.959078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:03.960075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:03.960515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:03.960937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:03.961209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:03.962306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 248/500] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:03.968121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:03.971682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:03.973154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:03.991721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:03.992669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:03.993574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:03.998081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 253/501] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:03.998949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.070449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36407a-8949-4fc8-ae9a-1061b077c9d4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.080489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.102746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae0929-8431-4ffd-a8da-bf3f7a0b80e9 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.103576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.104362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.104641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.104913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.115370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.116967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:04.134550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:04.135586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 249/502] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:04.136684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.139523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.140392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.140682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.140958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.151909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:04.157805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:04.159862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:04.160741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:04.161535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 254/503] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:04.162291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.246498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a9db-c1df-4d08-b731-325ded1f1346 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0968b-4ce3-4d1c-811a-08cb1f9ee166 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.282358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.283152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.283446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.283836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.284651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:04.285029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.286793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451f2d8-6d50-4532-a164-b5ec3d42d637 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.293646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.294814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.295105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.295377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.295517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 250/504] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:04.296205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:04.296576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.296763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b57b-68e4-4bb5-ba46-f640109b4bae None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.297005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b57b-68e4-4bb5-ba46-f640109b4bae None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.297279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b57b-68e4-4bb5-ba46-f640109b4bae None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.298005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.302371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 251/505] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:04.302371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308b57b-68e4-4bb5-ba46-f640109b4bae None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.310737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.325802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 255/506] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:04.326903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.390899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.411706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308b57b-68e4-4bb5-ba46-f640109b4bae None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.433570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5de2a7-447b-44f6-878d-98596212eada None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.445503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.446163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.446484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.446766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.447490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:04.448051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.449417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.467970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 252/507] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:04.467970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.471689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.472355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.472634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.472908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.473650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:04.473955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.475374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115821fb-0478-4072-aa90-bde646b9dec5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.485069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 256/508] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:04.485236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.485535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.485788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.502358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:04.503940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:04.557260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.573012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 257/509] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:04.574063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.574219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6951-69fa-4d12-81e8-c306549f7ae7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.614587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.615402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.615755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.616034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.625008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.627285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.628867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:04.645929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:04.646870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:04.647699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:04.647699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:04.647699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.647699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:04.647699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 253/510] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:04.647966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.674372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53811f90-5f55-4cea-9b7a-7b8cfbcaa10a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.708378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.709018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.709298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.709579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.751643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1040b-25b4-4ac3-96a0-f1f389e7c127 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.768203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.782025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.783542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.786467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.787249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.787534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.787810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.795175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.796636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:04.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:04.818495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:04.819384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 254/511] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:04.819882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.874371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.897535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 258/512] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:04.898390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:04.922129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629527-4006-4dcc-b08e-2eed081d715a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:04.956931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:04.958746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:04.959265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:04.959644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:04.971280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:04.972742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:04.985665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:04.990559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:04.991868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 255/513] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:04.992990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.001523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748c2a-83d0-4b1b-a80a-ff81e47e228e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.038608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.039396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.039681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.039960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.050049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.051541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:05.068860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:05.070012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 259/514] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:05.071035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.108720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bedea4-e7e0-469a-b8ea-b7b5a40debfd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.111418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.146910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.147708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.148080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.148410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.149183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.149584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.151208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.174060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78b12a-b469-4176-85a9-e5c3aef28cac None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.175999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 256/515] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.176891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.208690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.209345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.209630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.209912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.210732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.211079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.212499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.227164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.231298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 260/516] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.232677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.296423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6430696-8ba7-442d-824b-4dea6c24d7aa None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.330150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.330825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27e20-a55a-47f7-824e-3ba2a6f0df70 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.330967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.331264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.331499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.342452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.345181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:05.364914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:05.366173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 257/517] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:05.367540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.376687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.377416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.377700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.378030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.379125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.379762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.381880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.385645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.394431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 261/518] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.395252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.464347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.473547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664276a-4d0d-48a8-8503-eff3f24c1778 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.504893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1a79-c5d4-4ce1-8b50-f5b0f9137130 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.514487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.515436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.515792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.516143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.517127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.517506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.519439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.534433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 258/519] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.535498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.543556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.544571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.544942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.545336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.555003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.556971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:05.577335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:05.578523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:05.579583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 262/520] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:05.580457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.610631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.639422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15156935-5e79-47d6-944e-fd9c29b4abb7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.678321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.679244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.679615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.680004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.681064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.681533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.683522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.702380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 259/521] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.702636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6a28-5b53-47dd-b825-f00188dc9a39 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.702731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.726100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.742088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.742993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.743346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.743692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.744517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:05.744897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.746357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.765278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 263/522] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:05.765278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.835130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c7c35-a56f-45be-a8b6-52bacebe9281 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.874567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721517-0bb7-4177-9abe-a398f2103f52 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:05.876854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.877657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.877938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.878214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.886165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.887643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.890398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:05.905317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:05.906395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:05.907871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:05.908650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.916693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:05.917373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:05.917657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:05.917938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:05.918682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:05.920673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:05.942129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 264/524] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:40:05 2022] POST /identity/v3/users => generated 247 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:05.943064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:05.993754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.061218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dfe5-402e-4182-b796-24d519f3aca2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.089635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e091ba-9171-4d96-8918-5159ecb29eb3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.111130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.111913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.112271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.112618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.114126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:06.114537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.116364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.135648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.136607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.137113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.137473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.147188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.148781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.158593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 261/525] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:06.159249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:06.167482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:06.168378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:06.169907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 265/526] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:06.171087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.225656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.277670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ae56-2ee1-4224-b3aa-7bb512cb80d8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.295821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.304359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876048-bdfb-4489-881f-ca7f362248de None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.320661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.321472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.321757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.322036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.323706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:06.324036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.325968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.350731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 262/527] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:06.351584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.358973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.362957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.363734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.364091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.449795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.451682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.482415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d845-9a7b-466e-89a7-d5ece37224cc None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.500813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.520672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.521712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.522195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.522723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.523706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:06.524214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.525956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.546322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 263/528] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:06.547573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.574095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.586481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 266/529] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:06.587643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.674690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc67e07-8507-4002-bf59-afde2709ec46 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.711188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133cabf-0bf1-425c-b222-2f1fdfa084d4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:06.726936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e405239-7306-4078-b70e-010d94be010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.727912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e405239-7306-4078-b70e-010d94be010b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.728424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e405239-7306-4078-b70e-010d94be010b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.728896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e405239-7306-4078-b70e-010d94be010b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.753207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:06.754225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:06.754838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:06.755425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:06.756917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=cd5dd586ede14a918301fd93b4c8f320) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:06.757441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.759174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.766294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 267/530] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/role_assignments?user.id=cd5dd586ede14a918301fd93b4c8f320&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:06.767448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:06.810942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.861415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e405239-7306-4078-b70e-010d94be010b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:06.863600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e405239-7306-4078-b70e-010d94be010b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:06.898728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:06.983583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b5fea-5561-4c3c-b936-fa7b81174637 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.036281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 264/531] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:07.041793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e405239-7306-4078-b70e-010d94be010b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.068028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.068754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.069039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.069312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.095178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.098496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:07.128623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:07.129567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:07.130510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:07.130510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:07.130510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:07.130510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:07.143343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 268/532] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:07.148219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.174430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:07.236506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e405239-7306-4078-b70e-010d94be010b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.273847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84e785-4f48-47de-9d5c-8ba3beb6608b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.288083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:07.314446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.315518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.316006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.316497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.317428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:07.317957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.319711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.320003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.320461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.320739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.321009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.334760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 269/533] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:07.340134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.346091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.355150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:07.389241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:07.390174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:07.391089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:07.391089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:07.391089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:07.391089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:07.401564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 265/534] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:07.402317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.484585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88243f-6c67-4b89-b915-2089aeca2fa4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.524028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.525163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.525644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.526112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.534952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.536659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:07.567406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:07.568316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:07.569161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:07.574049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:07.579932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.632305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d24e8-184d-478e-b0f7-569a254b2e3a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.642267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:07.687914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:07.705810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.706507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.706792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.707067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.707783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:07.708093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.709487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.728911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4895e-6931-4979-b67e-b7ecab61f168 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.729624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:07.739368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.771933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.773949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.775517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.776004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.777185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:07.778619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.781097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.798017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:07.805918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:07.921753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:07.933535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735409-c3cf-4d30-84c7-6f7f6faffa57 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.938939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ffc58-a629-4547-871d-ed91cdb07fce None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:07.977797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.978588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.978875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.978990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:07.979178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.979668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.979953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.980054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=2861607f39d44172851da17c72704358) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:07.980229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.980323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:07.982011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf99f2-931d-4190-9609-1c46845e3f9e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:07.990432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 272/538] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/role_assignments?user.id=2861607f39d44172851da17c72704358&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:07.992417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0169e-6ef3-4d96-b68b-1369605b1cfe None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:07.992417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0169e-6ef3-4d96-b68b-1369605b1cfe None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:07.992417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0169e-6ef3-4d96-b68b-1369605b1cfe None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:07.994381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 273/539] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:07.999855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:08.001745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:08.004032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.004852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.006876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:08.027214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:08.028169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:08.029079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:08.038809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 267/540] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:08.042803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:08.044748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:08.047574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.145610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:08.192145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 274/541] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:08.220154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f252-6d22-4bc3-ac8c-ef33fd18a52b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.226236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.278807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:08.279469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.279784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.280059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.280855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:08.281165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:08.282635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:08.304543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 268/542] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:08.310814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.389665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:08.411808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981f609-cc64-4d41-baf1-20d93651e0cd None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.413272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11e383-2c28-42c4-8644-ca99c0e60840 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.423046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:08.453297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:08.453863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.454148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.454459 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.455291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=30dea44d32f0405a9344f004640d44a4) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:08.455679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:08.457145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7f7c-74d8-46b8-90ae-b607b218f333 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:08.462559 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:08.463802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.464316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.464802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.466561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 269/543] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/role_assignments?user.id=30dea44d32f0405a9344f004640d44a4&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:08.468169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d3df-6535-454b-93b5-64dd1a1fe2ef None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.468677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d3df-6535-454b-93b5-64dd1a1fe2ef None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.469152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d3df-6535-454b-93b5-64dd1a1fe2ef None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.471054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 270/544] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:08.480276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0149-288f-4854-afc4-35095f213230 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.480927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0149-288f-4854-afc4-35095f213230 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.481404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b0149-288f-4854-afc4-35095f213230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.490852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:08.501045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:08.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:08.502856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:08.506704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 275/545] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:08.512188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.533579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-787b0149-288f-4854-afc4-35095f213230 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:08.534849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-787b0149-288f-4854-afc4-35095f213230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:08.660084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8c50-f6f8-40af-aa7a-0c8500337599 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.678155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 271/546] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:08.703117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:08.703847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.704078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.704357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:08.705110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:08.705421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:08.706855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:08.711538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b0149-288f-4854-afc4-35095f213230 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.716915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 276/547] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:08.724508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:08.746701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:08.913302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b0149-288f-4854-afc4-35095f213230 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.915945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:08.937352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134cc7-c81f-48e2-8cd5-44a347b5f0b5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:08.990913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:08.991692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:08.991976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:08.992250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.012686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.014014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.014295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.014587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.016057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:09.023818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:09.024728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:09.025531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:09.026788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 272/548] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:09.035662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.038240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.038715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:09.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:09.057130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:09.058695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:09.058695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:09.058695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:09.058695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:09.058695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 277/549] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:09.063407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.134747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.178397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a372b-1940-481e-9457-24d4522cda8a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.211236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3101-5d44-42bd-960e-c441e244e093 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.220274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.221376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.221665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.221945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.222701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:09.223373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.225106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ede14-bdc7-41a4-a124-5609f86760bd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.242509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 278/550] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:09.243822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b84-313c-4775-a3f8-779c700805d3 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.244115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b84-313c-4775-a3f8-779c700805d3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.244393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c6b84-313c-4775-a3f8-779c700805d3 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.245766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 279/551] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:09.247897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c6b84-313c-4775-a3f8-779c700805d3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.265614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.266295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.266795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.267077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.268450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:09.268777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.270356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b5f67-9d2b-4eca-aa3d-e9a322db7794 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.281274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 273/552] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:09.283144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.283585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.283996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.304334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:09.305840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:09.331294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.368108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c6b84-313c-4775-a3f8-779c700805d3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.390093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.399393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 274/553] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:09.400360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.416307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.417194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.417476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.417752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.435128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.436652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:09.457071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:09.459363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:09.461541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.518655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4afa2-b90a-41d0-88f9-f0dd7db41ea6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.561889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.562884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.563716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.564228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.566360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce46c45-06b0-473c-b748-0c50d5bfa61e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.577922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.580006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:09.602343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:09.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:09.605631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.610366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.610366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.610366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.610366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.616180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.623570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:09.631820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:09.634188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 281/556] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:09.636216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.695959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.726050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62bcce-1fb2-47fa-9a2a-181cddd74c6d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.762583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.763927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.764212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.764508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.765407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:09.765876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.767440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.779239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f0b8-ba08-432d-a087-1626e12a2d94 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.786671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 276/557] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:09.787592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.825874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.826587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.826875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.827156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.827889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:09.828202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.830200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.850289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 282/558] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:09.851314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.857913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.904746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e808369-4338-46ce-8034-2a7779cc44fd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:09.949549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:09.950357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:09.950692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:09.950991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:09.952380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:09.952741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:09.955208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:09.966859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 277/559] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:09.967531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:09.984493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:09.987831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c882-a424-426d-8964-11a2aef2feea None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.039033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.039761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.040034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.040298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.053712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.055189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:10.074269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:10.075842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 283/560] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:10.077377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.112184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fcb99-6570-4037-997d-1535addc02ea None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.148429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.159441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.160317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.160691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.161028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] PATH_INFO: `/v3/users/swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.169380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.170973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:10.186308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:10.187431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:10.188666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 278/561] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:10.193088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.194733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ae001-9114-40ee-b5a5-4bed3f79889a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.246145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.246956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.247769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.248039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.259706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.261176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:10.280079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:10.281033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:10.281922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:10.281922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:10.281922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:10.281922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:10.285987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 284/562] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:10.286897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.291368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.314590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd1b86-9174-4901-ba0a-b9361f5c3fce None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.350900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.351563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.351848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.352125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.352849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:10.353159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.354581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.371864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 279/563] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:10.371864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.394471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c620cc-6cc8-440b-929c-bc51f3786fd8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.422267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.433846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.435182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.435486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.435760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.436587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:10.436891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.438320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd99ef-1a02-4a6a-b31a-e22f4be6bd57 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.455737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 285/564] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:10.457003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475cdbc-08cc-4efd-92a5-e36b39229356 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.457291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475cdbc-08cc-4efd-92a5-e36b39229356 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.457564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475cdbc-08cc-4efd-92a5-e36b39229356 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.458979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 286/565] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:10.459466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475cdbc-08cc-4efd-92a5-e36b39229356 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.487146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b436f-a0e7-4154-bb4c-49fbf0745d71 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.523194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.523976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.524256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.524529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.525245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:10.525547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.526001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.527707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.543765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 280/566] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:10.543765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.575137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475cdbc-08cc-4efd-92a5-e36b39229356 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.610592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.611387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.611673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.611951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.618933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.620370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9d6178e6-1855-4090-b3a1-5672efc3aa8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:10.634857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:10.636139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:10.637724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:10.640500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 287/567] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:10.641719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.642006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.642279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.646267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d044ac7-249f-449d-a48b-5c6197089a47 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.653900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.659263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:10.660593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:10.681409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.682050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.682348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.682634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.728024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 288/568] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:10.728837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.749392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.751057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8c1ff-ee42-4c0d-a58a-7e47d8f28ac0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.778987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.827267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba25a-4774-44ef-b430-ba2898938124 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.848715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 281/569] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:10.850022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43473f35-086e-4d2a-b175-933ca1ebd8e2 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.850310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43473f35-086e-4d2a-b175-933ca1ebd8e2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.850614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43473f35-086e-4d2a-b175-933ca1ebd8e2 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.852019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 282/570] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:10.852552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43473f35-086e-4d2a-b175-933ca1ebd8e2 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.860934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:10.865823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.866709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.866990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.867263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:10.875152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:10.876603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:10.894400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:10.895383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:10.896357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 289/571] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:10.897042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:10.963071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43473f35-086e-4d2a-b175-933ca1ebd8e2 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.996938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58dd9c-9be9-48e8-b574-19a9b3115fdc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:10.998500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:10.999163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:10.999447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:10.999727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.000509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:11.000821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.002260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.011030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.014645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 283/572] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:11.015455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.039332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.040181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.040489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.040787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.052664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d1358ea9-cc0f-4ab9-97ae-4f9f995ab168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:11.060969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:11.061908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:11.062771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 290/573] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:11.063469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.063711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.063947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.080027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:11.081153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:11.083064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.118613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17f19f-b5e0-4e53-bd9a-f11d1ab980c7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.146307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 291/574] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:11.147239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.160366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.161317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.161979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.162259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.173430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.174917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:11.192030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:11.193130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 284/575] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:11.194052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.246972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d29f14-c040-4a01-b502-cbfd464467b4 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.249069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.281908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.282606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.282892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.283171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.283996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:11.284313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.285737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.293546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702bb99-f7a8-472d-9f95-ad6d503b9c41 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.303114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 292/576] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:11.303910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.314916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.332482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.333301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.333583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.333858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.347742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=c064d58c15704009864fc43a1e630676, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.349200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.361810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 285/577] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:11 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:11.362625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.413114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41cbf1-6bfe-4684-b256-77a620de6fd6 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.454440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.455121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.455402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.455676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.456603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:11.457181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.458659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.475052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 293/578] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:11.475052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.505909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1f53d-3407-4753-878f-e137290f4c36 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.507261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.551041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.551839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.552121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.552395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.564013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:11.571673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:11.572902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 286/579] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:11.574710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.601860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8a1ae-b3eb-4043-ada4-69f278da01de None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.606242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.639936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.640605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.640890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.641172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.641889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:11.642202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.643602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.655885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 294/580] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:11.659244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.685487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73b2f-1732-4c85-83c0-ef5696542dc4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.720301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.721339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.721339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.721664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.732900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.734378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:11.752001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:11.752993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:11.753879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:11.753879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:11.753879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.753879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:11.753879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 287/581] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:11.754152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.764044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.783754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb3f0-90ce-4304-a55e-f8e698b1fb4b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.820686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.821508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.821791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.822074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.838248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.839741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.842995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:11.858289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:11.859210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:11.860096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:11.860096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:11.860096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:11.860096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:11.860096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 295/582] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:11.860376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.869821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd75cb-5fb2-44c8-9ff6-db1fe3a20b22 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:11.906801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:11.907474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:11.907754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:11.908027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:11.908770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:11.909074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:11.910457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:11.918623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 288/583] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:11.923691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:11.975729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af4416-ffd9-4a06-93a2-b7b40b546df8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.020325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.021264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.021546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.022125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.024641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.027550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4160891-a96b-4a0d-afd7-911976734238 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.032795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.034237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:12.053607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:12.054728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:12.055821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:12.056548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 296/584] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:12.057371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.067065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.068049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.068531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.069019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.069935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.070477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.072059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.090106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 289/585] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.091726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.094633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.161852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee70bd3-d100-4299-9aee-4a72194fbdc2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.196618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aecbcb-6480-4473-9a2e-2ad0ea64d719 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.196924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.197843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.198123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.198458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.199200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.199507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.201403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.214367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 297/586] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.214528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.240793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.241590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.241874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.242151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.253627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.255116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.273520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:12.274747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:12.276078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:12.277260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:12.277260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:12.277260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:12.277260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:12.277260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 290/587] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:12.277554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.329436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b57d7-1c97-4de4-b46e-e3bb374350c6 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.335624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.366428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.367092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.367378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.367654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.368432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.368740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.370181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.381999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 298/588] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.382841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.391608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91833181-5da8-4b80-9926-37cd7c9b4452 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.430243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.431245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.431693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.432095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.439731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.441437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:12.456287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:12.457356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 291/589] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:12.458275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.494076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f9190-93ec-49af-98c1-9e10d02a760f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.510600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.537866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.538832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.539121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.539402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.548291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.549772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:12.564847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:12.570808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 299/590] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:12.572037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.579869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef9b21-53bf-435c-8e06-1b2adf24fc14 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.597299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.621178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.622141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.622629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.623070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.623953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.624432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.625990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.638567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 292/591] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.639552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.679738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504686f-cd4b-4033-b7fa-538bc34a9c93 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.717738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.718649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.719075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.719505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.720321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=3ec0b4d695474375b2f87727d41e40f7) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.720628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.722039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.728004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 300/592] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/role_assignments?user.id=3ec0b4d695474375b2f87727d41e40f7&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.728788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.755359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74103fba-1e5a-4a7a-826e-426addc750b9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.793604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.794288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.794604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.794875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.795666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.795974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.797426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.801103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.811358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 293/593] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.812223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.849891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:12.891129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b109482-57f5-40da-8b70-861cb620c572 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.929037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.929867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.930150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.930765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.931877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:12.932188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.933142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc89f61-f711-4661-b89f-81f1be467938 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:12.933615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:12.959635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 301/594] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:12.960477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:12.977640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:12.978450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:12.978733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:12.979007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:12.986808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:12.990151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:13.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:13.004893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:13.005836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 294/595] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:13.006444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.076444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.085195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90685-a32d-41b1-b40a-46c2a9cfda78 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.104665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.111543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b27fe-4532-4c78-b407-eb5b5d6a6f4f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.125967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.127128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.127563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.127990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/cd5dd586ede14a918301fd93b4c8f320/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.145421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=cd5dd586ede14a918301fd93b4c8f320, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.146832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.147069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.147975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.148399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.148811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.160559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.162379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.166429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 302/596] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:12 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/cd5dd586ede14a918301fd93b4c8f320/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:13.168251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:13.203520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:13.204688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:13.205677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:13.205677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:13.205677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:13.205677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:13.207727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 295/597] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:13.213966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.377875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e54e3-58b9-4c72-bbe9-38f867336c15 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.387707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46a44-9fdf-46e5-9657-ef747f5ff08c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.413167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.413935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.414224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.414527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.415251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:13.415559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.418585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.426223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.426930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.427211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.427485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.428322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:13.428627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.430211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.440820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 296/598] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:13.441737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.451231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 303/599] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:13.459400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.465362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.467265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.579998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9190ab-7e32-46af-a856-9d700f97ed0c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.584356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e2ef-b85e-4f86-a535-37c303f883ae None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.617779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.618610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.618894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.619169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.621038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.622101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.622623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.623109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.626209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.627812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.634290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.635981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-7cf47f89-713a-47b8-adb5-348610d43f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:13.642425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:13.643321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:13.644206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 297/600] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:13.645311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90ec1-a35b-4581-a68f-b98a9e9a5711 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.645601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90ec1-a35b-4581-a68f-b98a9e9a5711 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.645878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec90ec1-a35b-4581-a68f-b98a9e9a5711 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.651571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 298/601] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:13.656125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec90ec1-a35b-4581-a68f-b98a9e9a5711 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-04fe7efc-1e88-4403-9198-ecfe197ad57c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:13.663561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:13.664430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:13.665320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:13.665320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:13.665320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:13.665320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:13.667518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 304/602] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:13.668949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.669459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.669936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.689752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:13.691605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:13.728058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.748725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:13.771597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec90ec1-a35b-4581-a68f-b98a9e9a5711 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.775731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 305/603] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:13.776561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.826060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.827671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.827964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.828247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.829109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:13.829423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.830951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.850528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 299/604] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:13.851382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:13.917839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300d70d-40d5-4b2b-b622-2082e9acfd75 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.963178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:13.964539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:13.965103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:13.965396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:13.966734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:13.967050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:13.967550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313bebc-6e67-40fe-8765-f27b7138cfd7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:13.968482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:13.992340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 306/605] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:13.993627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.007023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.017535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.018580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.019027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.019443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.031436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:14.039418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:14.040523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:14.041374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:14.043596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 300/606] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:14.044543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.058278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.130385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4b442-3a88-4613-be0f-01a237f0e4d9 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.184118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087ec-c5e4-4f87-9643-cb0d6e88b89e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.189252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.190356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.190815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.191262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/2861607f39d44172851da17c72704358/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.207291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=2861607f39d44172851da17c72704358, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.208953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.226693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 307/607] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:13 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/2861607f39d44172851da17c72704358/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:14.227795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.249411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.250629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.251235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.251719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.269092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.273533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:14.306639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:14.307723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:14.308624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:14.313756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 301/608] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:14.314971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.384042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.393003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfb929-0375-423d-9995-1e2e2d270b02 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.433209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.433884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.434222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.435005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.435758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:14.436061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.437488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.440027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.447781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 308/609] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:14.454841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.557938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1cdc-c416-4c4c-9cdd-3746c2f4d0d5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.563916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345568c-4564-44e1-a617-e5f0cc557bd5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.596781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.597745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.598144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.598539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.600158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.600980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.601406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.601904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.602968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:14.603484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.605140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.611024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.612463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.625868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 302/610] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:14.632283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:14.649345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:14.650287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:14.651535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:14.651535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:14.651535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:14.651535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:14.655148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 309/611] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:14.661406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:14.679578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.800466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76e32b-48b6-4341-bdef-ff9ee1e6d3af None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.816431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:14.847775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.849222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.849736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.850246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/30dea44d32f0405a9344f004640d44a4/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.868832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=30dea44d32f0405a9344f004640d44a4, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.870662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.878541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f7b0-0b7b-406d-9344-a5d10d9c716b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:14.888857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 303/612] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:14 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/30dea44d32f0405a9344f004640d44a4/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:40:14.924203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:14.925273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:14.925762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:14.926291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:14.927284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:14.927829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:14.929444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cc7ba-5cd6-4d0d-9461-4697b7c8024c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:14.944335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 310/613] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:14.955010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.012873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7e90d-82dd-42c9-be59-58c8745e7b5f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.013159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7e90d-82dd-42c9-be59-58c8745e7b5f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.013440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7e90d-82dd-42c9-be59-58c8745e7b5f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.014988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 311/614] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:15.020444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.020725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.020996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.063513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:15.064719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:15.109923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4862e-c0f8-49d5-8795-5c7033617f05 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.157793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.158624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.158903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.159175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.168576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:15.170092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:15.171957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.177388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 312/615] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:15.186258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:15.187420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:15.188571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 304/616] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:15.196013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.209002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.227849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.325204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8618ec-8cf2-4b6b-8a6b-bb78d7575623 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.329328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79eb6f-a904-4f1d-ad5d-fde80fe55b37 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.393305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.393977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.394259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.394555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.396382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.397172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.397448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.398190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:15.398521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:15.399961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:15.401587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.420802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:15.433399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:15.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:15.435396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 305/617] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:15.440054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 313/618] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:15.440700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.445626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.514909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.582165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299542-b930-464b-b63e-889e2c00f2f1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.596736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.625673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144fd7d-0d51-4fc2-8aad-482b98a73ecf None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.625893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.625893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.625893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.625893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.656622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.657412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.657765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.658113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.658971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:15.659344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:15.660803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88079-9e82-4fd4-8946-76ee339c8beb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:15.670314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 306/619] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:15.671834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dbc1a-eb4c-49d7-86ac-2668392b0db9 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.672197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dbc1a-eb4c-49d7-86ac-2668392b0db9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.672572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dbc1a-eb4c-49d7-86ac-2668392b0db9 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.677999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 307/620] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:15.679682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728866-7f92-4307-9476-867558b8136f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.679972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728866-7f92-4307-9476-867558b8136f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.680293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728866-7f92-4307-9476-867558b8136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.698003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-99728866-7f92-4307-9476-867558b8136f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:15.699294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-99728866-7f92-4307-9476-867558b8136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:15.707236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:15.708782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:15.773721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.786447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 308/621] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:15.787644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728866-7f92-4307-9476-867558b8136f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.821675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 314/622] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:15.827292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:15.881750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:15.926786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728866-7f92-4307-9476-867558b8136f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.954701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf3661-a678-455e-9e8e-3034a4e4aeb4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:15.971624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:15.973692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:15.974074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:15.974530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:15.988834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:15.990297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.008820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:16.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.009851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.010799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:16.010799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:16.010799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.010799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:16.011034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.011034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.011034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.011220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 309/623] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:16.011884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.025750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.027589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:16.052820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.054069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.055745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:16.055745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:16.055745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.055745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:16.058418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 315/624] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:16.059390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.081645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.128911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a25dd0-717b-4f0b-963f-8386a3aca4e1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.151389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.179084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.181041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.181458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.181938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.196374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:16.206884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.208126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:16.209839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 310/625] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:16.211250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba801-5fbe-403f-ab4a-a09cbb8160a2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.211381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.254025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.254857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.255207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.255551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.256417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:16.256788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.258296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.273283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 316/626] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:16.275836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.356934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b8139-919d-42b3-bd74-70645fa58e38 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.392988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.415632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.416529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.417072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.417541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.418924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:16.419888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.422062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.433402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61dae0-34fe-4a41-972a-ec99721a32db None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.449503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 311/627] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:16.449503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.482206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.483557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.484034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.484477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.485386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:16.485852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.487428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.498970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 317/628] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:16.500080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.536133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.630609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf88132-6c34-4b34-929f-c962bdce70e0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.643692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf95f35-0b65-424b-ab1f-bf8ad6f4eb75 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.683389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.689922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832550-9967-4251-beda-257e06d10005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.691083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832550-9967-4251-beda-257e06d10005 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.691567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832550-9967-4251-beda-257e06d10005 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.692166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832550-9967-4251-beda-257e06d10005 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.694051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.695436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.696015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.696466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.701356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b832550-9967-4251-beda-257e06d10005 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.703152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b832550-9967-4251-beda-257e06d10005 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.709619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.712357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8b832550-9967-4251-beda-257e06d10005 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:16.718146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.720342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:16.722629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:16.724382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 312/629] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:16.724983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832550-9967-4251-beda-257e06d10005 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-8c048996-22db-44a5-8a7b-9b3e996e9460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:16.740159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.742694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:16.745179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:16.750084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 318/630] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:16.752456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64b752-46d8-4c32-bbd8-dd92f4ff3ce8 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.753040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64b752-46d8-4c32-bbd8-dd92f4ff3ce8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.753472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64b752-46d8-4c32-bbd8-dd92f4ff3ce8 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.758398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 319/631] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:16.758398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64b752-46d8-4c32-bbd8-dd92f4ff3ce8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.828907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:16.832372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832550-9967-4251-beda-257e06d10005 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.868421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.869210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.869496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.869777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.881104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.882588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:16.901811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:16.903729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 313/632] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:16.905615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:16.942879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64b752-46d8-4c32-bbd8-dd92f4ff3ce8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:16.983474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:16.984165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:16.984453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:16.984733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:16.985604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:16.985924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:16.987550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089ed7-bd54-48df-9f55-00ca5917458e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.007646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 320/633] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.012148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.013233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.013708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.026402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfc6f5-3ed2-43c3-b084-b39dc2470baa None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.035790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:17.037621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:17.068184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.068831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.069145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.069463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.070243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.070574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.072014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.089356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 314/634] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.089356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.096090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.101638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.134355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 321/635] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:17.134355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.212491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ccf63-284b-4bc7-a7be-b8d3dec340e9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.238459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0852f-4089-4954-a857-779f9b5e4aec None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.252883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.253460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.253756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.254015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.254784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.255067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.256557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.272738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 315/636] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.272738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.282787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.283716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.284070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.284408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.296076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.297505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:17.316167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:17.317232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 322/637] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:17.318153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.351678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.354540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.395367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6452e4-2409-42fd-aafc-bcd1ffbf92a2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.430003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73cc1a-5b04-4d08-8e3a-24ad1d51ad98 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.431909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.432579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.432872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.433151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.433980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.434320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.436924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.450764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 316/638] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.452113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.479071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.480197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.480676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.482140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.496020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:17.504342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:17.505396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:17.506264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 323/639] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:17.507670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.611048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a951-1aa6-436e-b6ff-aea8b9749ffd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.642159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.656944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3f1fe-10b8-46c7-91f1-2fe94dfcff0a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.658621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.659596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.659757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.660008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.663076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.668477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.671085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:17.696030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:17.698225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 317/640] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:17.701313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.707260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.707260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.707260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.707260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.708081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.708081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.708081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfca24-3b99-4024-bc0f-377fe2bef51b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.726298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 324/641] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.726298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc048cd-d679-4baf-a024-c1fb1877223b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.726298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc048cd-d679-4baf-a024-c1fb1877223b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.726298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc048cd-d679-4baf-a024-c1fb1877223b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.726298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 325/642] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:17.727276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc048cd-d679-4baf-a024-c1fb1877223b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.824076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112f725-4251-41c9-9e0d-6ffc76a1c66e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.834934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc048cd-d679-4baf-a024-c1fb1877223b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:17.861058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.861843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.862129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.862439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.863192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.863668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.865239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.869595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:17.870268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.870910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.870910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.871728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:17.871984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:17.873409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1842c8c-69fa-46ed-8778-a1c517bd46c8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:17.878860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 318/643] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.878860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:17.893202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 326/644] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:17.894915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:17.895422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:17.895910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:17.912797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:17.913985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:17.916739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:17.951424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.006308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 327/645] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:18.006308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.012986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf7471-0c36-40e9-b30a-97c2af80745d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.051496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.052689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.053178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.053643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.061620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.063553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:18.080143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:18.081046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:18.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:18.088548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 319/646] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:18.088548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.112354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81505aa-928a-4cde-a32d-faff0fb9df4a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.148147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.148986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.149375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.149731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.159350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.162366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.164791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:18.185575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:18.186560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:18.187490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:18.187490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:18.187490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:18.187490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:18.187490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 328/647] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:18.188675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.201214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4fa9-cf5f-42ef-a068-eef25cf558a5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.219334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.238268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.239063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.239350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.239753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.302252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe987-63fa-463d-9b29-1c215433b18f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.320978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.323146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.339369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.340092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.340412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.340690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.351373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:18.359982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:18.361709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 329/648] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:18.363397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.424111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.472534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 320/649] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:18.472534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.488109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98affa-8052-4ee4-8923-3c39e42953eb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.492668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.529969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.530838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.531220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.532007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.532544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:18.533000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.534931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.552001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 330/650] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:18.552892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.587617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a1f5-958b-4d03-a4b4-62ccd9f4e828 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.627983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.629017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.629358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.630381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.631279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:18.631718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.634247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.657299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 321/651] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:18.658679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.674505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c9790-3dcc-432c-9edd-233fa83e2898 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.692018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.714958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.715847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.716193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.716479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.717235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:18.717548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.718954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.727805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 331/652] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:18.728677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.750288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:18.774085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9f83e-ce31-423a-ac00-f2735ddbdac0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.813840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.814764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.815050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.815329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.826293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.827770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.840372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dadd0-8466-4059-ac69-5901adc5dc30 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:18.845551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:18.846476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:18.847485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:18.847485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:18.847485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:18.847485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:18.850640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 322/653] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:18.852932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.878517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:18.879661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:18.880113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:18.880406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/3ec0b4d695474375b2f87727d41e40f7/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:18.899910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=3ec0b4d695474375b2f87727d41e40f7, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:18.901390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:18.917914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 332/654] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:18 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/3ec0b4d695474375b2f87727d41e40f7/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:18.920693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:18.972469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7d76-043c-4cdd-b83a-c3ab32a6f8e2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:18.973529 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.019703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.028489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.029721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.029721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.029950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] PATH_INFO: `/v3/users/glance` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.038860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.040354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:19.057273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:19.058429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:19.059444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 323/655] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:19.060044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.097460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6a73b-af24-4f16-b6e1-46377ba55315 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.144016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.144901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.145423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.145649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.159840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.161564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:19.182541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:19.183704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:19.185396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:19.185396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:19.185396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:19.185396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:19.187387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 333/656] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:19.188414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.217178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d662a33-4e6d-48e3-b49b-542a2d99b28c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.267107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.267779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.268065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.268617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.269354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:19.269665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.272105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.282565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.290777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 324/657] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:19.292757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.319336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffb3-aa56-4d19-a583-3b320f07cc79 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.357872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.358905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.359376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.359837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.360921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:19.361442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.363107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.363958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.387899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 334/658] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:19.387899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.431214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc033490-6fb2-4c9f-a45c-133fb9ff2899 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.481949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.483155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.484236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.484524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.485254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:19.485565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.488366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.505629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 325/659] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:19.506877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.507596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc19ec-cc2c-4414-aec4-e3eea0ba8c54 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.546218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.547480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.547764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.548036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.551193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.561534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.563148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:19.579087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:19.580048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:19.580931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:19.585089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 335/660] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:19.585944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.643833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693900-14ed-49a5-a608-f1cab2d4effb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.655216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.680967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.682541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.682541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.682541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.703434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bc767-acb8-40d7-af04-61beb841742e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.744756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.745557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.745838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.746108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.754141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.755622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.764207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.766618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47406a1-dd12-483d-a505-aac6b8845376 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:19.769713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:19.770983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 336/661] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:19.772419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.838051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.889281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 326/662] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:19.889884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e86b5-60f2-4e93-9b3b-d9aa1cc62f1a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:19.890983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017bae-2d10-4bde-a8c7-1d125d1fd031 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.891299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017bae-2d10-4bde-a8c7-1d125d1fd031 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.891583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017bae-2d10-4bde-a8c7-1d125d1fd031 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.893348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 327/663] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:19.899128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8017bae-2d10-4bde-a8c7-1d125d1fd031 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:19.914134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:19.931665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:19.932601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:19.933046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:19.933458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:19.934444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:19.934926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:19.936507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:19.954430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 337/664] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:19.955332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.026925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8017bae-2d10-4bde-a8c7-1d125d1fd031 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.070851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.071636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.072047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.072847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.073894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:20.074528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.076779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f633c30-afc9-49b2-9fc9-90c1480cbee5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.082221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2928-6320-461f-9bcd-6c78c3804c79 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.097618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 328/665] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:20.099067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.099359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.099633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.108078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.117372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:20.118650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:20.139704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.140508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.140816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.141067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.152059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.154207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:20.172638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:20.173611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:20.174597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:20.174597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:20.174597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:20.174597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:20.178399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 338/666] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:20.179313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.197836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.204777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 329/667] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:20.205666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.295628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf902d3-ee29-4f92-a326-25643b60718b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.316999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa59f43-7abf-44da-a25b-dd90d0adbaed None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.332834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.333492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.333773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.334046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.334906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=cd5dd586ede14a918301fd93b4c8f320) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:20.335218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.336644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.343261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 339/668] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/role_assignments?user.id=cd5dd586ede14a918301fd93b4c8f320&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:20.344064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.368687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.370054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.370564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.371028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.378597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.458232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.462580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.464167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.464934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7af61-f7ed-4892-a6c1-7752a8cb50ee None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.506992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.507886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.508414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.508414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.509255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:20.510363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.511199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.527970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 340/669] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:20.529009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.601524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 330/670] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:20.603154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.665614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.686165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d506f9-4452-44a6-8efa-7289585f7e5f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.727124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.727847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.728133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.728478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.740130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:20.747901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:20.748766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:20.749902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:20.750902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 341/671] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:20.751642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.758709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e174052-85a6-45fb-a14f-81cb4b675a3e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.797811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.798805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.799193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.800278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.807521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.809721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:20.825101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:20.826457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:20.827746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:20.829393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 331/672] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:20.830306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:20.897210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575effff-c7c1-4d4c-9a5d-b33842205199 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.949896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.951235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592f09-228a-4dde-abe2-02247719e238 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:20.951633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.952252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.953238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.976722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.979630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:20.992473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:20.993321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:20.993702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:20.994096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:20.995079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:20.995561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:20.997790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:21.005913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:21.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:21.008169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:21.008169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:21.008169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:21.008169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:21.010073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 342/673] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:21.011128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.014703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.014916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 332/674] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.016820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.040359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.138050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caceb4-7b91-4174-b189-485f8dff22e0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.143250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a46dd2-aad0-4cbb-b77f-16bd566e1c26 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.174622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.175276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.175560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.175834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.176555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:21.176861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.178224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.185488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.186403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.186694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.186965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.187770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:21.188072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.189530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156fe5e-db9f-4da3-8e8f-d8dec4f7cf7f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.192958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 333/675] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.193543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.211774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 343/676] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.213368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56a8bd-03d4-4e98-84fd-69a119f452df None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.213667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56a8bd-03d4-4e98-84fd-69a119f452df None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.213932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56a8bd-03d4-4e98-84fd-69a119f452df None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.219661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 344/677] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:21.219661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56a8bd-03d4-4e98-84fd-69a119f452df None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.296563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.300252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.324518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5685a-f9fd-43e2-9675-1008743bfaf4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.348516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56a8bd-03d4-4e98-84fd-69a119f452df None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.372314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.373098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.373374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.373644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.384394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.385847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.390056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.390994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.391294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.391620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.400358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.401852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:21.405740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:21.406658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:21.407560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:21.407560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:21.407560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:21.407560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:21.410480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 334/678] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:21.411540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cf45153b-ba82-425a-96ff-cecf4ba2f980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:21.416408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:21.417346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:21.418375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 345/679] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:21.423985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.423985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.423985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.441525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:21.443117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:21.520004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2ab9-84bc-4cbf-a43b-9b85ce833080 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.523276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 346/680] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:21.524124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.563337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.569853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.570693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.570928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.571271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.572021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=2861607f39d44172851da17c72704358) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:21.572592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.574246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.586758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.590743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 335/681] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/role_assignments?user.id=2861607f39d44172851da17c72704358&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.591566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.666234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c41bf-ef93-4f57-b7cd-350e3311d516 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.721153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200b2bf-965a-4193-aa26-4bb90cd0098e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.725253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.726281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.726803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.727389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.728347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:21.728887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.732975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.753715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 347/682] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.754662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.769980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.771065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.771472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.771753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.772666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:21.772975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:21.774452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:21.788780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 336/683] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:21.789916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.862229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.886171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9c646-2e00-4792-92d6-f24c1ed2bb31 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.900418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:21.930010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.931711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.932184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.932642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:21.933663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadae822-996c-4e4e-af78-dbf9b9b446a3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:21.948950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:21.958859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:21.961018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:21.962270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:21.965622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 348/684] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:21.968361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:21.993360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:21.994267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:21.995037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:21.995037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.003530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.005527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:22.020530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:22.021619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:22.022844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 337/685] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:22.028312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.092538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd98b36-3ff7-41fc-a3f4-f90391e2b6f2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.139330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.140700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.140876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ff07b-edff-4011-bf24-6d1099a9848a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.141530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.142060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.144364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=30dea44d32f0405a9344f004640d44a4) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:22.144880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.146751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.156239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 349/686] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:21 2022] GET /identity/v3/role_assignments?user.id=30dea44d32f0405a9344f004640d44a4&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:22.158070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.160354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.183289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.184540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.184891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.185386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.186559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:22.187001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.188543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.199317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 338/687] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:22.205903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.214867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.311579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7accec7-ce3b-4b08-b50a-67bfd1a89dee None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.365127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.366124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.366616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.368367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.369382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:22.371996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.374188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.416003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 350/688] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:22.424470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.428590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb65b8-9224-4975-91a7-7316ddf02f71 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.479396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.480408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.480907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.481260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.496604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.498542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:22.523693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:22.524859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:22.526262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:22.526262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:22.526262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:22.526262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:22.526262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 339/689] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:22.536563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.552938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.562275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b5d7e-c05a-494b-b8ea-b3f810c6eb18 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.578634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.606812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.607595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.607913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.608271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.661305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0339fe-bf56-4ba8-85e5-e063011abc68 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.701146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.702647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c1dbf-9e39-46ce-a73e-896bf5aa5b0a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.705319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.706094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.706403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.706681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.707519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:22.707986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.709152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad376b-eea9-4c22-9f4c-b44c60f94c57 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.723817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 340/690] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:22.725238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c2bd-3bd2-4faa-9d6f-c6919d72eabd None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.725629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c2bd-3bd2-4faa-9d6f-c6919d72eabd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.725908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015c2bd-3bd2-4faa-9d6f-c6919d72eabd None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.727506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 341/691] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:22.731131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015c2bd-3bd2-4faa-9d6f-c6919d72eabd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.826232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.834930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 351/692] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:22.836472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.836830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.837197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.846919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:22.858469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015c2bd-3bd2-4faa-9d6f-c6919d72eabd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:22.859293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:22.860580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:22.897285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:22.898103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:22.898433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:22.898704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:22.906739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:22.908248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:22.924944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:22.927338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:22.929629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:22.934764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 342/693] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:22.934764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:22.941798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 352/694] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:22.942569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.050109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974336ab-89a9-4093-ad67-1065fbed18d7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.085630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.086823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.086951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bc56b-59ef-4648-8761-1403808c7025 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.087426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.087892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.098589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.100345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:23.118705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:23.119748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:23.121033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:23.121033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:23.121033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:23.121033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:23.121268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.121418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 343/695] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:23.122441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.125786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.128797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.129740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.130223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.130699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.131705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:23.132828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.134509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.154959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 353/696] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:23.156135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.229041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cc053-becb-4661-93be-c0d6e051768f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.264411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.265683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.266036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb8e3f-474a-493b-b2cf-075677428de1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.266223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.266815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.267937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.270178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:23.297581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:23.298664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:23.299507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:23.302529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 344/697] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:23.303633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.315805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.317682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.318252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.318894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.320152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:23.320580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.322792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.339656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 354/698] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:23.341164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.400132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.410978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.442276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774631d7-edee-44cc-b9c9-f9a153d4f97c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.475177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e4435-860a-49eb-af6d-155829c1cf90 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.485055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.486246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.486578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.486860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.526318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.527483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.528558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.529205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.530186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:23.530806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.532543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.538856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 355/699] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:23.540011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.584890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.588058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.670240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54691-bb92-4d06-bd08-9ab1da1f326f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.678810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.700753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:23.727289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.728502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.728943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.729379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.740573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.742641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.758255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 345/700] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:23.761082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:23.762501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:23.763505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:23.765909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 356/701] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:23.766056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.771626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.886756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addedc01-e76c-41f4-9421-238c77e38eb0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.929839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:23.930560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:23.930831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:23.931108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:23.931827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:23.932124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:23.933501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:23.942797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 346/702] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:23.943352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:23.967827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a01ac-fd52-4bdf-ba96-0daf632b5236 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:23.995510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.053417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.055252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.055909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.056718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.059257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:24.059984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.063282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5646c-bbbb-4b09-a51b-2ac4955b9aaf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.080638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154ef59-39e8-4574-8ead-d780951816e3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.092493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 357/703] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:24.094978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea61f-41e9-4f24-bb27-00a0a3dcec23 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.095631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea61f-41e9-4f24-bb27-00a0a3dcec23 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.096237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea61f-41e9-4f24-bb27-00a0a3dcec23 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.099724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 358/704] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:24.101353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ea61f-41e9-4f24-bb27-00a0a3dcec23 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.126428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.127566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.127858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.128136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.142894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.144332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.165523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-f5f8b1d5-8555-408a-bbfd-272aad62292e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:24.166217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:24.167734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:24.169334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:24.169334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:24.169334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:24.169334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:24.172137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 347/705] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:24.173432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.173724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.174007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.194661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:24.195875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:24.271980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ea61f-41e9-4f24-bb27-00a0a3dcec23 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.293144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 348/706] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:24.294522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.324453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.333139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.334136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.334670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.335151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.336281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:24.336846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.338525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.345205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 359/707] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:24.346347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.447442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.461262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e22c0-ee3e-48fc-a952-504da016d192 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.479673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19555f-631a-449b-a56d-7ad0e8f08128 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.501750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.502952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.503456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.503942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.525026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.527142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:24.547175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:24.548135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:24.549029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:24.549029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:24.549029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:24.549029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:24.551485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 360/708] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:24.551774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.552451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.552734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.553007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.553730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:24.554044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.562414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.568105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.574207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 349/709] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:24.580239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.662910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.690377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef90e5-2dcd-4d31-b805-7af700cb2aea None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.728595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.730527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.730527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.730527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.738490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.740151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.751257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada81d6-165f-4639-824b-0bebed44f1f8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-fe235940-b970-4471-9192-9a9721e42c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:24.756964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:24.757833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:24.759257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 350/710] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:24.760175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c85bd-842a-44a1-872f-0cddfcd7b4b8 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.760175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c85bd-842a-44a1-872f-0cddfcd7b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.760415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c85bd-842a-44a1-872f-0cddfcd7b4b8 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.761750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 351/711] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:24.766800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c85bd-842a-44a1-872f-0cddfcd7b4b8 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.797058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.798170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.798170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.799265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.800156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:24.800156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.802965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a3a08-c917-46bd-a1ea-9342627b5ff6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.815252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 361/712] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:24.816753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.817053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.817335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.834687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.839911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:24.841948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:24.878172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c85bd-842a-44a1-872f-0cddfcd7b4b8 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:24.910527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:24.916762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:24.917629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:24.917966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:24.918363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:24.919253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:24.919636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:24.920362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 362/713] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:24.921427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:24.927271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:24.951060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 352/714] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:24.957021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:25.076216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305282a4-48ce-46a4-ba40-d154f060146b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:25.098467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a87c-225b-4af3-a021-f651c2afef4e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:25.113812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:25.118564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:25.119628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:25.120161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:25.120624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:25.122151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:25.123983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:25.144040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:25.145106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:25.145406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:25.145792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:25.146745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:25.148317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:25.149774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 363/715] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:40:24 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:25.150960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:25.177399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:25.178582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:25.179503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:25.186702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 353/716] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:24 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:25.193563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:25.327972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:25.343809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07662c7-fe6d-4626-b22f-760a08cdadf5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:25.392134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:25.393382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:25.393822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:25.394265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:25.395286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=3ec0b4d695474375b2f87727d41e40f7) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:25.395488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a1e74-afca-4747-b87c-5e953dd542e3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:25.395948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:25.398641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:25.408625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 364/717] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:25 2022] GET /identity/v3/role_assignments?user.id=3ec0b4d695474375b2f87727d41e40f7&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:25.442481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:25.443656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:25.444173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:25.444687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:25.445650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] RBAC: Adding query filter params (name=network) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:25.446199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:25.448158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2908fe-5e7b-44ce-8f77-5311998d42be None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:25.454126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 354/718] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:25.459337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:25.522657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:25.588600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1ae2-5d3e-4b53-ba37-5d55a4282ba1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:25.625695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:25.626383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:25.626671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:25.626943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:25.627646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] RBAC: Adding query filter params (type=network) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:25.627949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:25.629304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357fb23-2bab-4af5-a948-dac173778feb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:25.635626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 365/719] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:26.156907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f4a6-0fec-4d48-ba56-9fa48a4d84a7 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:26.157928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f4a6-0fec-4d48-ba56-9fa48a4d84a7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:26.158440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48f4a6-0fec-4d48-ba56-9fa48a4d84a7 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:26.160304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 355/720] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:26.165955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d57b-80ba-49b7-b770-168a6d04a6c9 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:26.166848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d57b-80ba-49b7-b770-168a6d04a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:26.167330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10d57b-80ba-49b7-b770-168a6d04a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:26.191394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4f10d57b-80ba-49b7-b770-168a6d04a6c9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:26.193020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4f10d57b-80ba-49b7-b770-168a6d04a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:26.288387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 366/721] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:26.320955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48f4a6-0fec-4d48-ba56-9fa48a4d84a7 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:26.476111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48f4a6-0fec-4d48-ba56-9fa48a4d84a7 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:26.503986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:26.529712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:26.530404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:26.530694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:26.530972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:26.531713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:26.533090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27867080-1771-4aa5-a074-4792fa94b400 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:26.555950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 356/722] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:26 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:27.133846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a042e-f55f-47cb-b1ae-1299437d40c1 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.134589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a042e-f55f-47cb-b1ae-1299437d40c1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.135081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a042e-f55f-47cb-b1ae-1299437d40c1 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.136920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 367/723] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:27.142583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.143135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.143617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.164029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:27.165974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:27.265052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 357/724] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:27.300730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a042e-f55f-47cb-b1ae-1299437d40c1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:27.373917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:27.452065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a042e-f55f-47cb-b1ae-1299437d40c1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:27.499073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:27.499875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.500159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.500434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.514017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:27.521821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:27.522810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:27.523704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 368/725] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:27.531482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:27.686306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabbd0c-0836-4668-bdae-bcbac2b5f966 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:27.697469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:27.726843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:27.727514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.727802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.728079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.728833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:27.729144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:27.730585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f5a0-7625-4903-b41b-0c0127fbb50f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:27.739908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 358/726] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:27.747414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:27.768357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b679-5c47-4e07-8b46-a3f8131f1174 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.768665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b679-5c47-4e07-8b46-a3f8131f1174 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.768909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8b679-5c47-4e07-8b46-a3f8131f1174 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.771521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 359/727] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:27.775613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:27.775838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:27.776118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:27.804333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:27.805542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:27.934901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 360/728] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 17:40:27.938965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12d702-d0f6-40a4-9292-5287431fc61b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:27.946704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:27.976249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.008633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.009798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.010309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.010817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.034648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.037101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:28.060789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:28.061744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:28.063041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:28.063041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:28.063041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:28.063041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:28.065462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 369/729] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:28.073549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.152165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.157667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915a8ea-e93b-479f-8aaa-43d0a4f10f72 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.197475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.199781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.199781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.199781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.210519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:28.214758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae922c6-df9c-4ef0-bcf1-84439043eb53 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-3614a0fd-e2c8-46b1-9e2c-1b870fd5b2eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:28.219021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:28.226551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 361/730] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:28.229981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6eb2d-d761-4f39-82a1-3d18464451aa None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6eb2d-d761-4f39-82a1-3d18464451aa None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6eb2d-d761-4f39-82a1-3d18464451aa None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 362/731] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0786d16-77d7-416f-a80f-ae478ad63c4c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0786d16-77d7-416f-a80f-ae478ad63c4c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0786d16-77d7-416f-a80f-ae478ad63c4c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.233409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 363/732] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:28.234692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.234692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.234692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.254073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:28.255476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:28.270027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.270883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.271198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.271452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.272186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:28.272498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.274232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b820e09-0243-4d5b-a961-cb626060e002 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.295130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 370/733] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:28.295130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.295130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.295130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.311975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:28.313429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:28.329087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.340603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 364/734] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:28.343198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.400890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 371/735] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 17:40:28.402077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.440340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.455918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8004f-43a7-4bbd-a56b-0d15d2dfb797 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.494349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.495011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.495291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.495571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.496942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:28.497365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.499064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.508752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 365/736] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:28.513771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.545269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6a1f-9317-48da-ab6b-81e49d96b2c0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.587284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.588424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.588890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.589346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.597104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.598856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:28.613824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:28.614854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:28.616060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:28.617963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 372/737] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:28.618894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.630590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.638973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f83c81-0639-4daa-9953-5dd422e4f8cf None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.675634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.676298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.676594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.676877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.682519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.685256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.699403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 366/738] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:28 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:28.700052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.716627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.747079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bad94-1fe7-4de4-bf4f-317ffe122c34 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.804531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.805867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.806447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.807070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.820952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:28.825719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad7eac-ead4-4d4e-abf5-14ee06309bc5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:28.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:28.831880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:28.832749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:28.835739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 373/739] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:28.837032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.867960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:28.868764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:28.869046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:28.869322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:28.881135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:28.882660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:28.902078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:28.903618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:28.904539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:28.904539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:28.904539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:28.904539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:28.904539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 367/740] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:28.905245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:28.913083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:28.960135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9336b2-afa0-4fee-99b5-2f7e73882621 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:28.999746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.000594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.001734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.002184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.002599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.003595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.003974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.005532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.029093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 374/741] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.033640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.065707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1a899-d2ce-43f3-b73d-5aa0bcda32a3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.111589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.113283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.113779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.114585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.115698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.116158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.118052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.130976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 368/742] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.132062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.161435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bb0ea-3057-4839-be4b-3860c08fe741 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.193612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.211540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.212579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.213067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.213550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.215615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.216134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.218184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.234612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 375/743] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.235368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.241981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d515f50-9c47-40d5-98a5-79eeeca39095 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.276907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.280356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.281163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.281498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.281788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.293375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.295095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:29.313944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:29.315023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:29.316088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:29.316088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:29.316088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:29.316088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:29.320193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 369/744] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:29.321113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.370241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831c56c-98bd-48da-83fa-29630ea36e3c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.414612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.415955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.416312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.416660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.427743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.428001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6acf-833f-4f05-805b-d06f78c635e5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.429245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:29.448137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:29.449772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 376/745] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:29.450741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.467833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.468678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.468996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.469275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.477599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.479202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.486609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-165f5e57-da5d-4ceb-9e53-28ca1c0fe87a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:29.494745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:29.495843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:29.497367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 370/746] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:29.498521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8cc0d-711e-4814-8a1a-9522bf4c9477 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.498521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8cc0d-711e-4814-8a1a-9522bf4c9477 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.499133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8cc0d-711e-4814-8a1a-9522bf4c9477 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.500263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 371/747] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:29.504038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8cc0d-711e-4814-8a1a-9522bf4c9477 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.548230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.570464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83787-bfab-40c9-a7be-cc45cad0ca77 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.608583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.609637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.609964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.610246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.611065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.611859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.613373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.624871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8cc0d-711e-4814-8a1a-9522bf4c9477 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.626667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 377/748] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.627483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.667032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.667746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.668061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.668467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.669182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.669492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.671010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1199de5-6430-41cf-b8cc-aa1f73a14c7e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.687600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 372/749] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.687600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.687600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.687600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.704382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:29.705897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:29.745755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.755535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f12415-2d7e-40c5-a9b5-36266012d3bb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.786688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 373/750] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:29.787541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.813277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.814986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.815432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.815865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.817217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:29.817706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.820078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.820352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:29.850925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 378/751] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:29.850925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:29.914476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa2a0-1f21-480a-a3bc-6a7452d0edc8 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.957203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:29.958057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:29.958361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:29.958679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:29.966113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:29.967608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:29.972664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1e07-553b-49d5-8aed-98a03f8614e4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:29.982412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:29.983550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 374/752] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:29.984480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.024417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.025498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.025784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.026521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.037120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.038812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:30.055792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:30.057014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 379/753] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:30.058185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.084244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.105858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.124193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e6c7b-265d-46b0-bff1-e32a6eb3b7bd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.163315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.164447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.164741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.165008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.165663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.167058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:30.181402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:30.182417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:30.183327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 375/754] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 17:40:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:30.184258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.200030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a58378-c3cc-4913-8258-012680134c60 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.242542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.243193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.243480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.243763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.305257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13bc6-ca1c-4bd5-ba20-2b3704cdcc60 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.321441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.323186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.341103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.350879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.351868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.352335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.352520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.353193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:30.353504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.355019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.368571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 376/755] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:30.374026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.387277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.448638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 380/756] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:40:30.449484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.497339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe0f3-625b-402f-b8f4-8a968706cfdb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.538958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.539631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.539914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.540193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.541016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:30.541317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.542779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.562384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 377/757] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:30.568370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.570187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773b1c-c4ea-4c3d-bdc7-7493ca7d755d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.587612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.613491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.614160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.614604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.614882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.615590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:30.615899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.617394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.622709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 381/758] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:30.627737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.646226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.685853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080f65-aba2-4b13-80ee-7e91a2112ae4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.727435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.728699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.728999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.729289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.731483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=c064d58c15704009864fc43a1e630676, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.733083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.747728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 378/759] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:30 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:30.748626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.767881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac7f40-ddb2-4c0a-99bb-8d373eada1a0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.813787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.815001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.815464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.815943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.830423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:30.832724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:30.855585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:30.856569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:30.857480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:30.857480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:30.857480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:30.857480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:30.861850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 382/760] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:30.862985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:30.896376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.906997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b32af7-a7e8-4745-9c34-fd1bdc4ed8cf None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:30.941181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:30.956943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:30.956943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:30.956943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:30.956943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:30.987340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2f21b-8f99-45c9-a561-b1b6152e6e29 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.028846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.029573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.029900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.030175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.030998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:31.031344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.032759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.032996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.035122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813459df-3044-4f86-8f62-087b08466e2b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.037935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 383/761] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:31.038770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.151401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 379/762] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:40:31.152372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843202-ddec-49e6-9b93-5326860852ac None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.152766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843202-ddec-49e6-9b93-5326860852ac None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.153130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843202-ddec-49e6-9b93-5326860852ac None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.155163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 380/763] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:31.156198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8843202-ddec-49e6-9b93-5326860852ac None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.162369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.173021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c570f-6397-4302-8abb-fda572f18cdc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.193076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.215991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.216597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.216878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.217162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.217961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:31.218314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.219799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.232424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 384/764] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:31.233092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.285819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8843202-ddec-49e6-9b93-5326860852ac None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.324961 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.325747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.325975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.326405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.327243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] RBAC: Adding query filter params (interface=public, service_id=e9caa1a426f4410b9519c8c66e2a3cc0, region_id=RegionOne) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:31.327597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.329112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851c2b7-9748-45d7-a468-e7a2ef9c74b2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.337574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 381/765] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/endpoints?service_id=e9caa1a426f4410b9519c8c66e2a3cc0&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:31.338928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.339217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.339550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.353685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ad4be-378f-479b-abf5-f82f3a3209ed None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.358136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:31.359378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:31.393144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.394008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.394268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.394606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.407149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.408727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:31.427899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:31.428892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:31.429798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:31.429798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:31.429798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:31.429798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:31.429798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 385/766] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:31.430675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.434664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.444649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 382/767] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:31.444649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.468873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.558070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91288bd-2a94-4e3c-8a81-a9cb9ec987a7 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.560980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5ec37-c6a9-408b-b1f4-069562d86067 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.597042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.598137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.598509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.598865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.599360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.599601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.599702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.599915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.600595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.602001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.608109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.608245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 383/768] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:31.609049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.610037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:31.635958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:31.637003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:31.637922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:31.642372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 386/769] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:31.643331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.720514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.739984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c18f46-9ba5-41c0-8521-a3499d6c3438 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.744729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:31.792968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.793934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.794225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.794535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.795440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:31.796114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.798503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.803213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970a63-be6e-4cc9-a594-c21a29062547 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:31.823207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 384/770] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:31.827034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.852233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:31.853503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:31.853962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:31.854446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:31.855597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:31.857666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:31.876555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:31.877821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:31.884800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 387/771] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:31.885652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:31.970439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f64b-6dff-4f34-8e2b-edc054e50b8e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.013122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.013798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.014091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.014551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.015691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.016012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.017746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.025090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76039f4e-004b-43ac-9b3a-dab7b544c5d4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.037517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 385/772] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.037517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.050688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.061283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.073610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.074765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.075253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.075705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.084805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.086296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:32.109641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:32.111036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:32.111942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:32.115050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 388/773] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:32.116141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.158488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479704a8-ec73-4acf-b8d6-e7bb13025759 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.194263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.194939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.195222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.195498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.196200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] RBAC: Adding query filter params (name=network) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.196507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.198300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.207442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 386/774] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.207442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.251069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda21054-ab36-4524-806b-cbcb6c9e570f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.287162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.287864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.288153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.288428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.289339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=9425457963714156a39a1eced4c8f85f) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.289562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.291053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.301365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.307034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 389/775] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/role_assignments?user.id=9425457963714156a39a1eced4c8f85f&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.308350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.343230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c338-8b29-4843-a220-9bffa8909316 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.387513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.388646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.389169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.389656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.390778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.391518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.393660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.410816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 387/776] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.419485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.459553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.548873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c0c5-4f49-4ecb-b93a-e9c193c06ea4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.596159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.596825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.597107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.597388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.598497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] RBAC: Adding query filter params (type=network) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.598810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.600475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.602059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992f21-a21c-4d64-832b-d44a7f05ceb4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.608597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 390/777] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.619280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.669343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.683308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.684034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.684319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.684639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.685474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=3ec0b4d695474375b2f87727d41e40f7) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.685786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.687261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.693869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 388/778] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/role_assignments?user.id=3ec0b4d695474375b2f87727d41e40f7&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.799757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17aeb0-fbca-4c08-b594-2f87be2572f7 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:32.817212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:32.851519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:32.852185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.852465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.852735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.853526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] RBAC: Adding query filter params (interface=public, service_id=5a8a48d3e96240ea8e91bbbab5dacb68, region_id=RegionOne) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:32.853835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:32.855336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8e46-95f5-498b-88cb-22ea0715ecf4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:32.869434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 391/779] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/endpoints?service_id=5a8a48d3e96240ea8e91bbbab5dacb68&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:32.875779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:32.952172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bf6e-1b97-43f7-9371-9ca175c0de5f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.952498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bf6e-1b97-43f7-9371-9ca175c0de5f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.952735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24bf6e-1b97-43f7-9371-9ca175c0de5f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.954358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 392/780] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:32.959803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:32.960090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:32.960364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:32.983005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:32.985177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:33.026691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de41357-88fc-4ad5-82aa-6376d90b2d2b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:33.112551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:33.113137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:33.113422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:33.113695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:33.114403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:33.115778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:33.122490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 393/781] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:33.127577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 389/782] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:32 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:33.146857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:33.258766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de9d36-409b-4fd5-a886-ff4a9fddeb2a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:33.271534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:33.312367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:33.313188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:33.313469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:33.313739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:33.332253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:33.340686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:33.341770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:33.342693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 394/783] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:33.357222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:33.427805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:33.476331 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1b09-172c-4783-a9eb-e69316719fa5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:33.512705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:33.513381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:33.513668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:33.513945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:33.514732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:33.515043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:33.516419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:33.527744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 390/784] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:33.533259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:33.679544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e9e09-4b06-49bb-8227-4adcc3c3351c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:33.730693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:33.731854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:33.732335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:33.732731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:33.749846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:33.752090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:33.782742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:33.784858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:33.787060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:33.787060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:33.787060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:33.787060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:33.787060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 395/785] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:33.793773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:33.955523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef2ce8-ea0f-4d6b-a934-338536d40c03 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:33.995044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:34.019990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:34.020670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.020947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.021217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.021938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:34.022239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:34.023653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b9420-53df-43e8-9b5e-48c1925a9049 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:34.046958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 391/786] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:34.052412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:34.103520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:34.147472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96209f2-91df-42bf-b403-fe1b2e2fe6d4 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.147808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96209f2-91df-42bf-b403-fe1b2e2fe6d4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.148011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96209f2-91df-42bf-b403-fe1b2e2fe6d4 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.149589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 392/787] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:34.158188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.159206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.159487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.191999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:34.195019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:34.240692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f97041-21ee-4fb0-b2e6-7bd33b553f84 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:34.300713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:34.301666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.301951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.302224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.312154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:34.313789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:34.335308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:34.337457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:34.338679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 396/788] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:40:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:34.341311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 393/789] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:34.345008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:34.398968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:34.462812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:34.495696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce56069-10d5-4d78-aef1-3b76e7df8592 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:34.561154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:34.561828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.562113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.562407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.563284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:34.563597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:34.565118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:34.597557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735721e-4c91-4319-a7b9-571ba0569c62 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:34.611792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 397/790] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:40:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:34.616372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:34.618709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:34.675349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:34.676348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.676764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.677045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.677851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:34.679438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:34.694361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:34.695362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 17:40:34.696233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:34.700727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 394/791] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jun 30 17:40:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:34.703035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:34.770398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb40432-be8c-4998-9e2d-55c9b709c890 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:34.830358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:34.830358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:34.830358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:34.830358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:34.870371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:34.900747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:34.902457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.025549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 398/792] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:40:35.035646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.093436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2421-e7f8-4847-984d-a8b72cf62399 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.094924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.129728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.130454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.130738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.131013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.132105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:35.132105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.133418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37377e5-9515-4e3f-b0cd-a81a77cef970 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.143402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 395/793] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:35.144875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239875-bbb2-4e9c-8d6f-acfb6e4ef55b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.145166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239875-bbb2-4e9c-8d6f-acfb6e4ef55b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.145344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442453f9-6a2a-4b13-9456-b941eb700f98 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.145503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239875-bbb2-4e9c-8d6f-acfb6e4ef55b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.147333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 396/794] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:35.148117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08239875-bbb2-4e9c-8d6f-acfb6e4ef55b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.186660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.187469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.187761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.188039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.191566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.200485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.201981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6510cbf3-3909-4b0d-aa6b-b50b388daebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:35.223546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:35.224619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:35.225659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:35.225659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:35.225659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:35.225659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:35.225659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 399/795] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:35.227175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.227481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.230446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.247286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:35.248518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:35.259182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08239875-bbb2-4e9c-8d6f-acfb6e4ef55b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.294638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.295270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.295606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.295882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.296621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:35.296933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.298393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.308972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 397/796] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:35.309917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.325924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 400/797] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:35.326912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.352948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.417039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7f4f5-4e2b-42c3-af06-8af86dfdff45 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.419293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.444743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866f588-4705-47db-a555-7fe3f48e57b2 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.454298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.455016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.455303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.455579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.456304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:35.456612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.459650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.472121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 398/798] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:35.472948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.488513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.489181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.489471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.489751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.490523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.491917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0fd2b-897f-4c3d-bf1a-e2e21a3883ed None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.513749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 401/799] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:40:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:35.514991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff9d2e-f218-407b-9ecc-14bb14b9b672 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.515345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff9d2e-f218-407b-9ecc-14bb14b9b672 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.515664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff9d2e-f218-407b-9ecc-14bb14b9b672 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.517183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 402/800] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:35.522263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff9d2e-f218-407b-9ecc-14bb14b9b672 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.626296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33857b66-8691-4efb-b322-243bfa2e255c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.634407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.679109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff9d2e-f218-407b-9ecc-14bb14b9b672 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.687332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.688149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.688663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.689341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.703787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-9c58b6dc-fbfb-48c7-b9da-5dd77bd3a2ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:35.712274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:35.713468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:35.715153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 399/801] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:35.715860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.715860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.715860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.731270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.732076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.732412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.732685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.734423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:35.735634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:35.742016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.743567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.754220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-63806c3b-0509-45d1-a34a-4989d3f507e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:35.767241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:35.768384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:35.769380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 403/802] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:35.770915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5b79-533e-4121-a304-6a33d79b9ce6 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.771794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5b79-533e-4121-a304-6a33d79b9ce6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.771794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef5b79-533e-4121-a304-6a33d79b9ce6 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.779675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 404/803] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:35.779675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5b79-533e-4121-a304-6a33d79b9ce6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.824288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 400/804] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:35.825520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:35.902805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef5b79-533e-4121-a304-6a33d79b9ce6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.939862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95ad0-eea9-4a3b-90fc-d7c8614e36b5 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:35.940459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.941286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.941488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.941774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.943191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:35.943191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.944215 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cc5af-fa05-44d0-b573-a21fd34cd0d0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:35.945823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:35.957017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 405/805] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:35.957017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.957017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.957017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.980627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:35.981843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:35.991204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:35.991871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:35.992699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:35.992699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:35.993175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:35.993473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:35.994939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.013605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 401/806] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:36.014452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.033047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.073142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 406/807] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:36.074058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.141933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcd3f6-5b13-4df4-bf3b-7c8d10c4d9b0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.178972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.179676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.179972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.180229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.189237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8482233-5b7e-4683-92ed-8d2ca6b2f936 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.191055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:36.199060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:36.200091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 402/808] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:36.200973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.221294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.232809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.233607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.233893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.234172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.245526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.247021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:36.264948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:36.265913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:36.266867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:36.266867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:36.266867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:36.266867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:36.266867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 407/809] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:36.267154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.286280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.320949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f425d-2c47-4577-b4ec-a400c67c5399 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.358926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.359587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.359870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.360148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.360982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:36.361280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.362747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.369181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 403/810] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:36.369949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.396318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dec65-a408-45c4-96a4-8f97e823d30e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.452974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.453798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.454161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.454563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] PATH_INFO: `/v3/services/network` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.455444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.457360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:36.475219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:36.476836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 17:40:36.478449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:36.480572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 408/811] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:36.481805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.512496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cae8b4-d53b-4456-89bc-36adaf0fd0e6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.546137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.557650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.558942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.558942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.558942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.559630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:36.559935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.561310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.563764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.578369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 404/812] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:36.579486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.631152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffb720-2802-4f4f-b49f-f1bbf126295e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.682620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.683375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.683668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.684116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.684864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:36.685178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.687614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.707659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2ed6d-c267-4031-9b09-529e77434a0c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.756502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.757196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.757505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.757783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.758571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] RBAC: Adding query filter params (name=network) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:36.758888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.760319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.767048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 405/813] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:36.767978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.842819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.849857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 409/814] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:36.852365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:36.893270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa225a-a9f8-4a79-b6a7-bc4896ffe4b1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.943857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:36.945306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:36.945713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:36.945997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:36.966058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:36.967678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:36.970727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a812-b6fc-4b0f-9f3a-58c513715117 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:36.989291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:36.990258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:36.991203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:36.991203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:36.991203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:36.991203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:36.992682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:36.995384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 406/815] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:36.996818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.021266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.022291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.022829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.023206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.024015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] RBAC: Adding query filter params (type=network) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:37.024422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.025899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.036129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 410/816] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:37.037175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.122742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ef078-fc43-4f0a-9815-89ecd52a5c74 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.154355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.155051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458832-2d55-429a-91db-a0861b2bba03 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.179027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.179847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.180127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.180705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.195369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.197817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.200815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.201545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.201903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.202268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.203058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:37.203366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.204741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.217568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 411/817] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:37.218314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:37.218778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:37.219720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:37.220691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:37.224478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 407/818] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:37.225484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.258676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.346117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f4d6-5fb8-48b9-9ff8-1f501e0f8952 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.358587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cca3a-a6c8-48ed-ba26-25948b1b39aa None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.388097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.389617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.390423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.390856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.397612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.398448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.398835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.399209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.400057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.401520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.402294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.403908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:37.423497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:37.424731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:37.427835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:37.430463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 408/819] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:37.430927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 412/820] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:40:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:37.431883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.436929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.481491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.578168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.617903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86514a-2b43-4323-b8a8-2e10d5b231dd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.627213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a207245-d4fe-451d-a9ec-601022b232cd None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.679401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.680126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.680427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.680701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.681668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:37.682018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.688899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.709972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 413/821] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:37.717671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.718403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.718615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.718720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.718958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.719814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:37.720121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.721580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.775951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 409/822] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:37.781756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.879797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7ce6e-54c5-4fd0-a642-8f02a2008f50 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.907087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.918506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c1cf-d018-4714-9942-d3de425bd9ed None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:37.924744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.925797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.926287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.926866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.930113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:37.939650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.941103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.958539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:37.959337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.959626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.959906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-fcddec44-a66d-4ca2-a0d8-5e6d2c23df9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:37.960234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:37.961099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:37.962025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:37.962025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:37.962025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:37.962025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:37.962025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 414/823] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:37.963081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd09210-fddd-4d9b-acf5-b54d507f7d4f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:37.963372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd09210-fddd-4d9b-acf5-b54d507f7d4f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:37.963641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd09210-fddd-4d9b-acf5-b54d507f7d4f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:37.964933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 415/824] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:37.967381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd09210-fddd-4d9b-acf5-b54d507f7d4f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:37.973116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:37.974629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-da4bace2-e997-403c-b51b-10eb1cbd42b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:37.997871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:37.998905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:38.000023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:38.000023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:38.000023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:38.000023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:38.001569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 410/825] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:38.006237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.006237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.006237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.022644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:38.023847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:38.086765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd09210-fddd-4d9b-acf5-b54d507f7d4f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.102577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 411/826] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:38.111471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.139933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.141204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.141492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.141931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.142555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.142888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.144372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.157376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 416/827] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.158235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.207141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.214797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.236383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55352b-0fa0-4cde-9cda-ccb9620c299d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.272087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.272754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.273039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.273317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.273686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246574ea-b43a-4b54-b4de-d459e7b7e4aa None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.274039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.274373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.275756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.287457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 412/828] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.293193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.310194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.311077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.311360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.311632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.312281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.313688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:38.326003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:38.327118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:38.327988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 417/829] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:38.328803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.404950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfaf87-141c-4e82-8191-87c75a32515b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.443624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf363e-16c3-403f-bfa6-30713138f31e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.444247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.445056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.445341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.445636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.448239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.455583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.457312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.462897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:38.473262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:38.474317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:38.475260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 413/830] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:38.475754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.493221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.494013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.494294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.494596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.502900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.504354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:38.519224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:38.520171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:38.521068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 418/831] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:38.521556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.585749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bdbb3-b06f-4135-bb58-f920e721ee42 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.622315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.623016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.623286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.623569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.624280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.624592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.625975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.635417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 414/832] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.635417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.635674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de69381-abda-40b6-867e-ce18f1d24206 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.672854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.673463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.673750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.674024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.674869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.675179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.676614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.689902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 419/833] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.690828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.701293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.727387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.748035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7d0af-3eb8-4c8b-80fa-71e21b5493f2 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.783385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.784049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.784332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.784616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.785411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.785722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.787205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.799443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fb04e-cc98-40ab-b012-de0d04eef342 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.799640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 415/834] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.800279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.842353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.843347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.843765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.844042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.845008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:38.845472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.847185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b1fa-6fab-4004-89d7-36e928c49bd4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:38.853001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 420/835] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:38.854360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9a42-27dd-4edf-80b2-ec3f0cc891d0 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.854668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9a42-27dd-4edf-80b2-ec3f0cc891d0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.854951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce9a42-27dd-4edf-80b2-ec3f0cc891d0 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.856544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 421/836] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:38.857094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce9a42-27dd-4edf-80b2-ec3f0cc891d0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:38.944678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cc5ad-558a-49c2-a257-b5dc01cf57d8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:38.966844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:38.990919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:38.992250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:38.992250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:38.992996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/3ec0b4d695474375b2f87727d41e40f7/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:38.994598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=3ec0b4d695474375b2f87727d41e40f7, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:38.996119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3f04e-b5e0-4014-8914-d7c77d754415 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:39.000418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:39.009119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce9a42-27dd-4edf-80b2-ec3f0cc891d0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:39.011567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 416/837] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:38 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/3ec0b4d695474375b2f87727d41e40f7/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:39.013698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:39.013984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:39.014293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:39.037278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:39.038570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:39.060518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:39.061821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:39.062315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:39.062903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/9425457963714156a39a1eced4c8f85f/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:39.087136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=9425457963714156a39a1eced4c8f85f, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:39.088606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:39.106511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 422/838] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:38 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/9425457963714156a39a1eced4c8f85f/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:39.141078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 417/839] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:39.166609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:39.294206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a939ec3-b795-4dee-9071-f516fef4474e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:39.336994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:39.338085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:39.338578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:39.339006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:39.339879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:39.341484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:39.355309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:39.357456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:39.359094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 423/840] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 17:40:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:39.364238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:39.478294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d0501-cf3e-4727-83b7-f28465085774 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:39.487866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:39.515624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:39.516621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:39.517310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:39.517590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:39.518306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:39.518645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:39.520030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57853f-6a89-4e81-80ea-3256f3d94256 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:39.525268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 418/841] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:39.531457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:39.545985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:39.640443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33319a-b2ca-4008-a167-f51fa45e3d66 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:39.677884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:39.678901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:39.679189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:39.679472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:39.680183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:39.680491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:39.681868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dbc9a-828e-490b-98ed-6092c4d84b09 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:39.687641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 424/842] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:41.006454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9659e-dbce-450c-87e4-0261678f78f3 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.006937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9659e-dbce-450c-87e4-0261678f78f3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.007018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9659e-dbce-450c-87e4-0261678f78f3 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.008699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 419/843] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:41.014293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53995d-e70b-4bcf-ab81-1515fc4da8ee None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.014711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53995d-e70b-4bcf-ab81-1515fc4da8ee None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.014880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53995d-e70b-4bcf-ab81-1515fc4da8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.032389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-df53995d-e70b-4bcf-ab81-1515fc4da8ee None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:41.033722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-df53995d-e70b-4bcf-ab81-1515fc4da8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:41.103062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 425/844] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:41.126009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9659e-dbce-450c-87e4-0261678f78f3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.131524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4deaf8-469e-4a65-89dd-bb7729f71e4f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.131812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4deaf8-469e-4a65-89dd-bb7729f71e4f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.132093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4deaf8-469e-4a65-89dd-bb7729f71e4f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.133720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 426/845] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:41.139424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.139774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.140049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.146243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:41.162661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:41.164696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:41.248816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9659e-dbce-450c-87e4-0261678f78f3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.254589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 427/846] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:41.274395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.297664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.301919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.301919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.301919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.301919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:41.310380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aaa39b-531e-42d3-a589-417f4cbabcb3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:41.338421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 420/847] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:41 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:41.367523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1729-30d5-48ce-a45f-caf575deea9c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.367905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1729-30d5-48ce-a45f-caf575deea9c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.368250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b1729-30d5-48ce-a45f-caf575deea9c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.374039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 421/848] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:41.375893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.376287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.376667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.397247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:41.398782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:41.428710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a7a06-36ac-4922-8a12-ee36f3db6abd None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.429886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:41.475992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:41.482261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.483898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.484278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.484651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.489052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 422/849] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:41.500363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:41.508524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:41.509547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:40:41.510447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:41.513691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.514181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 428/850] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:41.520465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.642361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e877c-755e-4f57-b0d9-a5d8c0d62bbc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.668490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448f8c5-1743-4053-9f2e-7352458153ef None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.685464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.686148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.686599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.686886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.687776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:41.688089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:41.689825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:41.704289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 429/851] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:41.711115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.718018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.718908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.719306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.719711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.734712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:41.742827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:41.743994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:41.750475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 423/852] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:41.754291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.780422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:41.785822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:41.849594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb0e9d-5c2f-4d73-b68f-2f23c1a4a95c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.885151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caebfa-7485-458d-9152-741e43717e0a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:41.890080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.890903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.891187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.891457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.903451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:41.904897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:41.921149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:41.921824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.922111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.922414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.923163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:41.923476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-27bf729c-f8d7-4b0b-a7f4-edf44aa612fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:41.924505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:41.925336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:41.926228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:41.926228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:41.926228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:41.926228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:41.926482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2efb4e-82c9-485b-be11-f574af9e402b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:41.930387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 430/853] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:41.931743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd46f7-a6e2-410f-a344-d9931942917f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.932028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd46f7-a6e2-410f-a344-d9931942917f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.932300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd46f7-a6e2-410f-a344-d9931942917f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.934470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 431/854] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:41.934963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd46f7-a6e2-410f-a344-d9931942917f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:41.939415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 424/855] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:41.940682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:41.940975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:41.941255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:41.961286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:41.962596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:42.046443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 425/856] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:42.046443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.057271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.059016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd46f7-a6e2-410f-a344-d9931942917f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.062767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.100801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.101465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.101746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.102019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.102888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:42.103306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.105257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.124675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 432/857] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:42.125566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.155998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65fc0f-7421-4323-be58-eed2dfb65d55 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.194613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.195401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.195847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.196131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.209863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.211501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:42.229659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:42.230654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:42.231568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:42.231568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:42.231568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:42.231568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:42.231803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d61af7-0bbf-4ffa-9e91-681d7a2e3d10 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.235566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 426/858] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:42.236174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.270639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.271308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.271604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.271862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.272594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.273966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.290035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 433/859] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:42 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:42.290722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.312049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.333225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.368322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75dd1b-91a9-4e64-aa6c-9e2c482f512e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.430066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.430939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.431226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.431756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.434576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e976e6-e5de-4cb9-b548-62e3b5ea42c1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.444570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.447883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:42.470573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:42.471759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 17:40:42.472921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:42.475752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 427/860] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:42.481241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.481656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.482372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.482654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.482930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.483663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:42.483972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.485389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.500879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 434/861] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:42.506851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.609446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a50ac-1360-4104-9357-9adc6f156e17 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.610960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.629585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.639964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1854d-1341-4770-947b-9ffa2a2c19aa None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.650444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.651436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.651899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.652389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.653229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:42.653536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.655007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.676300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 428/862] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:42.684291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.691263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.692347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.692829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.693250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.703898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.706523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:42.727140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:42.728155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:42.729099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 435/863] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:42.734763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.815004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf6fcf-ed47-41b3-897d-53420b7fbbde None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.857279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.858193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.858516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.858799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.872198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60eff-4577-406a-829a-6f9cd18bd2de None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:42.918968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:42.920552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:42.921083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:42.921927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:42.923438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:42.924043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.924712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.926181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:42.943544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:42.948810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 436/864] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:42.959985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:42.961603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:42.963513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.083886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f71e2-4970-4bb3-aeb2-b96c94d0cd78 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.096123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 429/865] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:40:43.101987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.129974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.130978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.131493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.131965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.181889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.208078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cecb3-601c-485a-ac6e-bd1a9dc89367 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.209226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.213653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.216748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.244261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.245096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.245349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.245616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.255868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.257377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:43.275508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:43.277025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:43.277919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:43.277919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:43.277919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:43.277919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:43.281914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 430/866] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:43.283624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.335135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 437/867] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:40:43.343783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.418619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17bdc1-0203-477e-9e20-9caf64fc5b08 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.454741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.455648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.457245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c2f8f-c184-40a7-ba14-eaf93c688e16 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.467077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.467742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.468031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.468302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.469015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:43.469321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.470713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.487430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 431/868] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:43.488781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.511872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.513205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.513958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.514234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.531160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.532937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-69537154-0929-43f4-81fb-cdbfb59f9e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:43.558539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:43.559524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:43.560439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:43.560439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:43.560439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:43.560439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:43.561199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 438/869] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:43.562552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e367e-2645-4963-bb0f-e564a2885d19 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.562841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e367e-2645-4963-bb0f-e564a2885d19 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.563118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e367e-2645-4963-bb0f-e564a2885d19 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.564395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 439/870] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:43.566710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e367e-2645-4963-bb0f-e564a2885d19 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.611471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5e545-771e-4736-8ca7-ee40485724f8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.647824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.648620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.648907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.649188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.656689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.658706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-3892f233-fd9e-4c26-86fb-498760d1abd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:43.673984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:43.674969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:43.675866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 432/871] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:43.676930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.676930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.676930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.683378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e367e-2645-4963-bb0f-e564a2885d19 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.694490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:43.695773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:43.721509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.722312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.722625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.722896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.723610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:43.723917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.725433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.741482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 440/872] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:43.741482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.746725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.755376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:43.773142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 433/873] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:43.773834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.848281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8c9c-6155-46b7-8d33-ae5adf9b9878 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.887741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.888397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.888677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8e021-b25d-42fe-b534-47bab4ec9785 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:43.888783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.888938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.889719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:43.890125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.891477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.909531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 441/874] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:43.909531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:43.937520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:43.938221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:43.938541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:43.938824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:43.946882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:43.948961 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:43.964182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:43.965145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:43.966131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 434/875] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:43.966647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.002737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.015455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.037399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f2b12-cb0f-4a81-85cc-ef31d98389eb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.077996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.079240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.079719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.080173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.080372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70abaf-7112-4cd4-b332-82c6ed346564 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.081167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.082735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:44.094673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:44.096272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.097125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 442/876] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:44.098093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.124672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.125409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.125764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.126118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.127031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=3ec0b4d695474375b2f87727d41e40f7) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.127401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.128935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.135949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 435/877] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:43 2022] GET /identity/v3/role_assignments?user.id=3ec0b4d695474375b2f87727d41e40f7&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:44.136777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.219172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed72f2b-98ff-4760-a60c-82dc24092dfb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.257226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff716844-8809-4b3e-a3c2-e3852f4305cd None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.258074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.259239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.259583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.260462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.260872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.262510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.264795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.274739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.278198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 443/878] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:44.285116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.303750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.304422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.304703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.305473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.306448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.306768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.309153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.315865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 436/879] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:44.322462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.416782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659a562-11d7-49e3-a7ac-7689a28a1f2b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.449409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bcb2b-2f08-41ae-a106-e4d3ae6a92b7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.459272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.460246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.460684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.461206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.462304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=9425457963714156a39a1eced4c8f85f) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.462955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.465198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8838-fc60-4ef9-be01-85846babe6ed None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.472131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 444/880] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/role_assignments?user.id=9425457963714156a39a1eced4c8f85f&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:44.491405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.492093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.492377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.492647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.493403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.493712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.495295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.496710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07df01-74d8-4622-a4ec-0f832a08c889 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.497000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07df01-74d8-4622-a4ec-0f832a08c889 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.497281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07df01-74d8-4622-a4ec-0f832a08c889 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.498940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 445/881] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:44.501707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 437/882] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:44.504483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.504772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.505049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.508398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.523727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:44.524957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:44.560351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.609290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.634728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 446/883] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:44.651146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.663946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ed4f0-6e13-4d8e-ac95-948cff05a584 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.720015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.720777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.720997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.721250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.722089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] RBAC: Adding query filter params (interface=public, service_id=e6889e4f94644073ab27a8e4b11711fd, region_id=RegionOne) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:44.722520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.723828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.731073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 438/884] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/endpoints?service_id=e6889e4f94644073ab27a8e4b11711fd&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:44.736139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.801263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c80ff-1a0c-4be2-a1e8-cc76e5e032ec None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.843643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520386-3381-467a-b737-01eb446b005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.844517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520386-3381-467a-b737-01eb446b005e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.844804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520386-3381-467a-b737-01eb446b005e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.845081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20520386-3381-467a-b737-01eb446b005e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.845743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20520386-3381-467a-b737-01eb446b005e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.847837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20520386-3381-467a-b737-01eb446b005e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-20520386-3381-467a-b737-01eb446b005e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:44.861542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:44.862590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:44.863663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:44.870402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:44.871288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 447/885] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:44.872998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23a82e-c66f-465b-a677-99e02fc1cf85 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:44.874219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520386-3381-467a-b737-01eb446b005e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:44.921304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:44.922023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:44.922305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:44.922614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:44.923305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:44.925418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:44.942361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 439/886] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:40:44.945614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:45.009627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20520386-3381-467a-b737-01eb446b005e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:45.051543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:45.052231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:45.052516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:45.052795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:45.053522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:45.053836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:45.055284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:45.061383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 448/887] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:45.068405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:45.191858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9818c-4e1b-42f0-b543-a4a0191c51d6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:45.229592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:45.230264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:45.230585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:45.230866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:45.231574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:45.231885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:45.233270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:45.239243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 440/888] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:45.245645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:45.372345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa634adb-1b30-445e-a89f-5c16fa67762a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:45.391807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:45.410267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:45.410957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:45.411248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:45.411526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:45.412321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] RBAC: Adding query filter params (interface=public, service_id=28e1cd1a7d324804a508b65c2e2b35db, region_id=RegionOne) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:45.412634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:45.414004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865004b-7a5c-4172-ab57-501b4c44d69d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:45.420234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 449/889] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:45 2022] GET /identity/v3/endpoints?service_id=28e1cd1a7d324804a508b65c2e2b35db&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:45.426803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:45.457714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:45.550445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b162c-f144-4e27-ac71-9cde69321b35 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:45.581237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:45.581914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:45.582202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:45.582508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:45.583199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:45.584569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f26d3-de1c-49df-bf3c-36b7f808ac1e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:45.591068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 441/890] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:45 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:40:46.427128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df746-f7ab-4fde-ad4d-b5c6231bdaef None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.427665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df746-f7ab-4fde-ad4d-b5c6231bdaef None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.427665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df746-f7ab-4fde-ad4d-b5c6231bdaef None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.429321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 450/891] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:46.434788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15364-f462-4e62-85e7-8fbdd2d3dbd5 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.435272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15364-f462-4e62-85e7-8fbdd2d3dbd5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.435351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b15364-f462-4e62-85e7-8fbdd2d3dbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.455541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e9b15364-f462-4e62-85e7-8fbdd2d3dbd5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:46.456735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e9b15364-f462-4e62-85e7-8fbdd2d3dbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:46.542446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 442/892] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:46.564577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df746-f7ab-4fde-ad4d-b5c6231bdaef None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:46.691392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8160e-d593-4819-8005-8d0f102b8e87 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.691855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8160e-d593-4819-8005-8d0f102b8e87 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.692044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8160e-d593-4819-8005-8d0f102b8e87 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.693713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 443/893] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:46.696065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df746-f7ab-4fde-ad4d-b5c6231bdaef None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:46.700243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.700522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.700798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.718325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:46.719658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:46.737259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:46.737789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:46.738639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.738916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.739183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.740002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:46.741260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:46.760042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:46.761720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:46.762613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:46.767311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 451/894] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:46.769329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:46.822457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 444/895] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:46.830702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:46.848139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:46.905043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6db0-faac-4b36-8a4d-5f7b0015de04 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:46.952433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:46.953022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:46.953346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:46.953629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:46.957143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:46.957143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:46.957143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:46.969968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 452/896] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:46.971672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:46.982184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369021c-d500-4e09-bad6-4e782ac94fcc None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.029142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.030229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.031025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.031318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.046670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:47.058144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:47.059421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:47.060370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:47.062419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 445/897] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:40:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:47.066114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.079023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.108031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee830b-47a7-49c0-becd-20e0d570ada6 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.121587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.156208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.157662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.158164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.159005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.160261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:47.160808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.162990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a0518-99af-48d3-ac59-039b03fdf626 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.182368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 453/898] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:47.199186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea6d74-0215-4359-ae48-5c9023535c8e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.205511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056325d-cef4-4fe3-a871-643ca1eb1eea None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.205703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056325d-cef4-4fe3-a871-643ca1eb1eea None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.205982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056325d-cef4-4fe3-a871-643ca1eb1eea None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.207862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 454/899] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:47.216058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.216558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.216864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.235773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:47.236994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:47.248821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.249770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.250275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.250719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.252072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:47.252511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.254857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.267387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 446/900] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:47.274053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.326728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 455/901] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:47.355591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.372308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.432469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147f6f-6ba2-433b-81d1-2a069c2a0fc5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.482470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.483960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.484244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.484520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.496777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.498242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.503261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31975663-b754-49e3-b4f3-a1856c95eedb None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.507426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:47.517636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:47.518690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:47.520044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:47.520044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:47.520044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:47.520044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:47.523942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 447/902] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:47.524922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.560207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.561622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.562545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.563281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.564132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.566942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:47.583030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:47.584161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 17:40:47.585035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:47.587883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 456/903] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:47.595488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.649284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b67e-81ea-43c7-af94-90cc9cf5d7b6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.678358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.696924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.697763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.698175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.698593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.699425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:47.699834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.701596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.726639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 448/904] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:47.727784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.743779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a046ab6-17a5-4ba5-9282-950abc11a16a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.795501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.796158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.796443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.796719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.797549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:47.798152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.799378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f358f26-1406-4333-afbb-8c327c14132d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.805209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 457/905] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:47.806611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9b7e-5ccb-4730-b937-36f96ca0080b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.806903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9b7e-5ccb-4730-b937-36f96ca0080b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.807185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada9b7e-5ccb-4730-b937-36f96ca0080b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.808646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 458/906] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:47.813197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada9b7e-5ccb-4730-b937-36f96ca0080b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:47.848484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:47.865309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdf8f3-041d-4027-858a-20c66c22f699 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.908612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.909434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.909722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.909997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.921340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.923096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:47.931001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada9b7e-5ccb-4730-b937-36f96ca0080b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-ae5bb697-5a88-41f4-aa01-54794b93c5aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:47.939539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:47.940755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:47.941842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 449/907] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:47.942530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.942530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.942672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.964710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:47.966196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:47.980760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:47.981507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:47.981750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:47.982063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:47.983595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:47.983595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:47.984604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.000563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 459/908] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:48.000563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.009789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.048976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 450/909] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:48.059985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.105863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.113621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76bdb9-769f-4ce7-9ce7-78a84985a5d4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.155483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.156176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.156481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.156735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.157564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:48.157893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.159353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.183429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 460/910] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:48.184285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.193680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5684bd-0ae8-4ece-b4db-7939301fd155 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.235833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.236293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.236581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.236860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.237632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.239046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.263434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 451/911] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:40:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:48.264919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.310072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.315688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95f8f4-d866-4784-9f6c-44a3119f4a77 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.377799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.378670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.378951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.379224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.380323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.381398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:48.396125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:48.397164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 17:40:48.398013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:48.402873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 461/912] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:48.404435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.412513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153b29d-97c3-4f47-ab3f-cfac71b8b2b1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.425171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.453704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.454373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.454663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.454934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.540365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.542202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4383c78-72b9-4c65-8be4-61efbf1ea125 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.542597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.578085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.591867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.592598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.592882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.593181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.594013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:48.594490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.597150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.609909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 462/913] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:48.611423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.662211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 452/914] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:40:48.668863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.700000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.727776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a516b-1b56-4bf6-b66a-a5c6ca7c9799 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.767233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.767899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.768185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.768460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.769166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:48.769471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.771022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.781416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 463/915] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:48.783008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.784484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf7da-c793-4983-b582-22e958a54113 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.815170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.825260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.826458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.826953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.827483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.839449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.841418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:48.862064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:48.863082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:48.863991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:48.863991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:48.863991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:48.863991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:48.866366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 453/916] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:48.878555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:48.912299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4b30c-50e7-4beb-8b0a-c86c756320dd None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.957030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:48.957698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:48.958026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:48.958305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:48.959090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:48.961033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:48.971386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:48.995267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2b01a-5263-4963-860f-6dbc2458fd79 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:48.998321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 464/917] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:40:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:49.056806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:49.057586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.057875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.058148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.059038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:49.059354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:49.060906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:49.076530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 454/918] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:49.082880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:49.204032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d768bd-5fcb-4ad6-9ab6-d1e6686dd190 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:49.241761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:49.242591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.242864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.243134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.250481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:49.251951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-59cecb2b-2f66-4104-b7d8-f018cac904f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:49.267613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:49.268613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:49.269518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 465/919] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:49.274923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:49.312002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:49.390007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ceb66b-4e4d-49b6-bec4-35dc61248be9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:49.419748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987372b-086c-487c-9296-a4e09ec9460c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.420061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987372b-086c-487c-9296-a4e09ec9460c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.420311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987372b-086c-487c-9296-a4e09ec9460c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.421945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 466/920] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:49.427216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:49.427398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.427681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.427868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.427972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.428152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.428427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.429212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:49.429562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:49.431196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:49.444710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 455/921] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:49.446942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:49.448704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:49.451076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:49.510048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:49.531683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 467/922] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:49.555393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:49.586747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:49.598957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fafc34-ce8c-4fc9-a819-d6eef38d9ad7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:49.639689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:49.640695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.641149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.641573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.642655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] RBAC: Adding query filter params (role.id=2ff5dd876493418c83374896107ec3b1, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=9425457963714156a39a1eced4c8f85f) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:49.643193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:49.644854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab679f-10fe-46ba-9bc0-c7710c0f9a5d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:49.652210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 456/923] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:49 2022] GET /identity/v3/role_assignments?user.id=9425457963714156a39a1eced4c8f85f&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=2ff5dd876493418c83374896107ec3b1 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:49.680508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578432c-6e22-4f91-a76c-352997d99dc4 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:49.721176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:49.721884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:49.722169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:49.722478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:49.723227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:49.724644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c83eba-4e81-401e-84a5-b3a43fe7904a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:49.746549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 468/924] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:49 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:40:50.535687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad47-586b-4eda-9090-fb1fae21c739 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:50.536418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad47-586b-4eda-9090-fb1fae21c739 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:50.536912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ad47-586b-4eda-9090-fb1fae21c739 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:50.538858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 457/925] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:50.544342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:50.544874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:50.545368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:50.563093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:50.565051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:50.612665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:50.651769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 469/926] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:50.675153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ad47-586b-4eda-9090-fb1fae21c739 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:50.785180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ad47-586b-4eda-9090-fb1fae21c739 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:50.825381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:50.826489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:50.826979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:50.827462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:50.828367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:50.829999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:50.831629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING keystone.server.flask.common [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 17:40:50.851364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.sql.core [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 17:40:50.851364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 17:40:50.851364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [parameters: {'id': '01acfba66d7c4155b92bcc981df46adc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 17:40:50.851364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91852) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01acfba66d7c4155b92bcc981df46adc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01acfba66d7c4155b92bcc981df46adc. Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:50.860570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 17:40:50.861454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application yield resource Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise self.value Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application session.commit() Jun 30 17:40:50.862505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.session.flush() Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application compat.raise_( Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 17:40:50.863429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 17:40:50.864351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:50.865155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [parameters: {'id': '01acfba66d7c4155b92bcc981df46adc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:50.866050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:50.868624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01acfba66d7c4155b92bcc981df46adc. Jun 30 17:40:50.871346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:50.878439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 458/927] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jun 30 17:40:50 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:50.880839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:51.015567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90fa2b-ea30-4958-b00c-d5baa2fe852f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:51.055157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:51.056376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.056868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.057941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.065524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:51.067285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-34a5b4dc-1382-449f-9b45-e6d4ac2ff9ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:51.082172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:51.083103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:51.084004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:51.090619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 470/928] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:51.092977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:51.122884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:51.206155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b7ae-8864-4f20-b79b-fbd5208bfc6f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.206472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b7ae-8864-4f20-b79b-fbd5208bfc6f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.206756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2b7ae-8864-4f20-b79b-fbd5208bfc6f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.208440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 471/929] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:51.211258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a572e-4a19-42e1-8bf8-be5421bd4b49 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:51.213664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.213952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.214234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.234756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:51.236802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:51.256378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:51.257568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.257852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.258416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.259227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:51.259561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:51.262432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:51.271808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:51.293635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 459/930] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:51.338155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 472/931] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:51.361202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:51.453110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:51.480707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e0ef4-4dd3-49f6-bb7f-954a5939da63 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:51.524818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:51.525635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.526298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.526609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.527283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:51.528671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:51.541310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:51.542938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:51.543785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 460/932] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:51.549561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:51.664943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4a096-28b5-4b4e-8d42-987c8fda5054 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:51.703412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:51.704091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.704376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.704654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.705369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:51.705680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:51.707093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf804c-565d-41e3-b5f2-b7abdb27e531 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:51.712756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 473/933] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:51.718491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:51.834610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda4f6a-601f-4ce9-9981-8b40d5705c38 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:51.853774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc84d3-e62c-4246-939d-e00baeed10a8 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.854801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc84d3-e62c-4246-939d-e00baeed10a8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.855081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc84d3-e62c-4246-939d-e00baeed10a8 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.856712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 474/934] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:51.861879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.862173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.862481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.874948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:51.875865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.876297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.876723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.877611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:51.878065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:51.879623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdccfc4-8d22-49b6-8ea4-cc27ef19c5ce None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:51.880575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:51.882031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:51.885929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 461/935] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:51.913184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254752-e57f-4c28-8cc7-8844cdf38e40 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.914711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254752-e57f-4c28-8cc7-8844cdf38e40 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.914711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40254752-e57f-4c28-8cc7-8844cdf38e40 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.915599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 462/936] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:51.923755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:51.926938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:51.927792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:51.928283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:51.946795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:51.950605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:51.958709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:51.986367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 475/937] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:52.009344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.050359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 463/938] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:52.071006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.145882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423289e-2fc9-42e5-9613-eeca4fc76e36 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.193785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b276-a36f-4b73-8dae-a17dcb902e6c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.201564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.202456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.202750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.203054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.219582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:52.230783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:52.231790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:52.232640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:52.236308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 476/939] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:52.241415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.241845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.249629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.250926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.251532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.253099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.255260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.257879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.263321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:52.275107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:52.276767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:52.277632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:52.279886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 464/940] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:52.287370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.385973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397a2e7-6639-4f49-b5e7-63020bc36b90 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.408326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32b359-9d08-4552-a81a-7b9a34404128 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.431051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.432063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.432553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.432913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.434225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:52.436736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.438227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.445351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.446070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.446741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.447098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.448415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:52.448806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.450306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.453294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 477/941] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:52.454250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.461503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 465/942] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:52.462601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.539570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.547312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.577175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03138-29da-4cc3-97cc-c8d1fcfde5f3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.593523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4273a-1095-492c-b17f-d1d886c63229 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.613425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.614250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.614568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.614846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.627537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.629039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.635594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.636643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.637132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.637589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.638612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:52.639113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.640671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.646819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 466/943] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:52.660249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:52.661250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:52.662168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:52.662168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:52.662168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:52.662168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:52.662433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.662508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 478/944] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:52.664876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.792253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d152ba5-300e-47c9-b90c-ed020719eede None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.809866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861554-f4f6-4331-9b44-659cc1e38e38 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:52.836912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.838059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.839915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.840923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:52.841738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.842320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.843306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:52.843861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.846931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.852431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:52.853094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:52.853382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:52.853662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:52.854512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] RBAC: Adding query filter params (interface=public, service_id=457c4b97694d41e49ab2ac635d518811, region_id=RegionOne) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:52.854847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:52.856213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:52.870427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 467/945] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/endpoints?service_id=457c4b97694d41e49ab2ac635d518811&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:52.873607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:52.874023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 479/946] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:52.874951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.012143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffdf4-1c18-4760-85e2-70414753dc8f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.013700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45af8e-903b-42ea-a60f-95743cc27691 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.055823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.056252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.057642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.057892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.058010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.058170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.058667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.059340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.060262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.061884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.067269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.068731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.074751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 468/947] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:53.085849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:53.086918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.087871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 480/948] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:53.093353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.164048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:53.211743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b101d96-826d-47e5-a1f9-4feefbb285fb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.268923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.269605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.269909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.270184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.271110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:53.271417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.272851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.303125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 469/949] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:53.309275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.363733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:53.417794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4065d9-327c-44bb-92a6-4254ba3824ec None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.453500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.454296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.454600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.454874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.466455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.467919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-b9f76996-e199-472b-9af2-db2bfe45a70a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:53.486858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:53.487861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:53.488755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:53.488755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:53.488755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:53.488755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.488755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 481/950] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:53.493986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.607061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6914a-a354-4757-98c9-679d1a6f6fc2 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.607366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6914a-a354-4757-98c9-679d1a6f6fc2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.607569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6914a-a354-4757-98c9-679d1a6f6fc2 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.609792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 482/951] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:40:53.617687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.618921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.619301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.624997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766cc03-3fa8-43e2-8016-0f6e4436443c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.640579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:53.645190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:53.646769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:53.674841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.675505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.675786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.676058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.676779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:53.677213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.678695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.692343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 470/952] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:53.698011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.734729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 483/953] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:53.757383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.798756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:53.807079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db104e-7e3f-414f-ba9c-b1e1b49d4a2f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.843820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.844776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.845064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.845336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.852742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.854260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:53.869923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:53.870894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:53.872002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 471/954] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:53.878104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:53.881363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec256e-c233-4214-b86b-04a40a4fd485 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:53.917174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:53.921226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:53.922222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:53.922818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:53.923300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:53.924253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:53.925938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:53.927651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING keystone.server.flask.common [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 17:40:53.941209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.sql.core [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 17:40:53.941209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 17:40:53.941209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [parameters: {'user_id': 'fa7db4162ea046dd965f74b9967463ed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 17:40:53.941209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91851) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:53.969000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:53.969994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application yield resource Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application next(self.gen) Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise self.value Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application session.commit() Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 17:40:53.971084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self.session.flush() Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application compat.raise_( Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 17:40:53.971983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 17:40:53.972880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application conn.query(q) Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application result.read() Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 17:40:53.973725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa7db4162ea046dd965f74b9967463ed', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:53.975843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 17:40:53.977882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 17:40:53.980123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:53.990062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 484/955] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jun 30 17:40:53 2022] POST /identity/v3/users => generated 160 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:53.991750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:54.007555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc8534-de70-4c6f-be31-1bc67cb90698 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:54.044992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:54.045502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.045792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.046068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.046901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:54.047272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:54.048927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:54.062387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 472/956] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:54.068328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:54.112258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:54.127144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067eb9f-b4c1-456c-bf01-9c184d15e91f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:54.164449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:54.165250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.165535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.165810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.174249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:54.176210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:54.189891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-fb5bcc6f-f965-485e-a5f1-879b9cb0cb0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:54.193744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:54.195183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:54.196416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 485/957] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:40:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:54.197056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675c58-5959-46d5-ab19-ae5248ee2af0 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.197056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675c58-5959-46d5-ab19-ae5248ee2af0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.197189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675c58-5959-46d5-ab19-ae5248ee2af0 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.201997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 486/958] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:54.205359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73675c58-5959-46d5-ab19-ae5248ee2af0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:54.211466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b113-6d5c-42b4-afd9-20b85c6ebde6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:54.258615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:54.259886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.260735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.261327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/9425457963714156a39a1eced4c8f85f/roles/2ff5dd876493418c83374896107ec3b1` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.263702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=9425457963714156a39a1eced4c8f85f, role_id=2ff5dd876493418c83374896107ec3b1)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:54.265320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429a4c3-4f85-489e-a292-28ed4750bfe8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:54.289797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 473/959] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:40:54 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/9425457963714156a39a1eced4c8f85f/roles/2ff5dd876493418c83374896107ec3b1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:40:54.289797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.289797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.289797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.308444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:54.309716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:54.368670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73675c58-5959-46d5-ab19-ae5248ee2af0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:54.407653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 474/960] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:54.418371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:54.418371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.418371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.418371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.418798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:54.418798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:54.419650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3101abc-bcdc-4c8f-94ae-e1436ec0eae3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:54.430547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:54.477343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 487/961] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jun 30 17:40:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:54.532460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:54.568482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb4f5-48c4-44e0-b80b-ce1dd4ea65ab None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:54.622277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:54.623471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:54.624022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:54.624807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:54.625801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:54.627510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5eb8a-f024-4d75-a4b3-7f06cf258e56 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:54.651107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 475/962] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:40:54 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:40:55.365723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f050e2-dfc1-4b48-8f16-4eb8e7041e23 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:55.366710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f050e2-dfc1-4b48-8f16-4eb8e7041e23 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:55.366710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f050e2-dfc1-4b48-8f16-4eb8e7041e23 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:55.367900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 488/963] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:55.373437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:55.373951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:55.374028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:55.392333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:55.393543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:55.473516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 476/964] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:55.496594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f050e2-dfc1-4b48-8f16-4eb8e7041e23 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:55.514843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:55.619324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f050e2-dfc1-4b48-8f16-4eb8e7041e23 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:55.658916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:55.660002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:55.660432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:55.660902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:55.661914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:55.663653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:55.676425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:55.677775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:55.678912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:55.686674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 489/965] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:55.688210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:55.776716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:55.816844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022eda28-52d8-4c9a-b344-3ff2fa826c5e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:55.853507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:55.854224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:55.854531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:55.854806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:55.855516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:55.855822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:55.857201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:55.863004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 477/966] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:55.868566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:55.981884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:56.021478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62016209-675c-4ca6-88d2-347609c89cc4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:56.069257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:56.069933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.070221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.070527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.071509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:56.071816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:56.073226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ac5e5-246e-4e96-8f73-3077265c2b11 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:56.079046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 490/967] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:56.084571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:56.203456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d95afe-a7c0-46fe-83a0-e07ab95518fa None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:56.233642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:56.240892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:56.241552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.241831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.242102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.243581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:56.245531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1603dbc-e496-4df4-8298-5c0e9df9d47d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:56.265134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 478/968] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:40:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:40:56.453497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31ef54-accf-4910-aac5-cde05f4a3410 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.454007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31ef54-accf-4910-aac5-cde05f4a3410 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.454084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31ef54-accf-4910-aac5-cde05f4a3410 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.455629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 491/969] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:56.461082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a0de-a22f-4b05-95ad-6b66a68b7523 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.461569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a0de-a22f-4b05-95ad-6b66a68b7523 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.461652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a0de-a22f-4b05-95ad-6b66a68b7523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.479857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3b31a0de-a22f-4b05-95ad-6b66a68b7523 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:56.481064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3b31a0de-a22f-4b05-95ad-6b66a68b7523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:56.566399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 479/970] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:40:56.587190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31ef54-accf-4910-aac5-cde05f4a3410 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:56.714075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31ef54-accf-4910-aac5-cde05f4a3410 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:56.752202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:56.754014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.754014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.754014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.762890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:56.767417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a1c67-8bc2-4e0e-9e2f-43063d69e91c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.767710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a1c67-8bc2-4e0e-9e2f-43063d69e91c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.767981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a1c67-8bc2-4e0e-9e2f-43063d69e91c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.768971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:40:56.769500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 480/971] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:56.774565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.774859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.775139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:40:56.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:56.778673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:56.779664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 492/972] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:40:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:56.784918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:56.795363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:56.796558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:56.878834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 481/973] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:56.904594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:56.917700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336a494-6a80-4c33-945e-62568ae45827 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:56.958617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:56.959299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:56.959590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:56.959867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:56.962396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:56.962714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:56.964726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:56.967704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:56.976467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 493/974] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:40:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:56.982911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.049810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4402ba-ad2c-4978-8896-e7a8c97aa1fe None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.062985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.091860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.092777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.093066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.093345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.094020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.095445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:57.107595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:57.108732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:40:57.109566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:57.113962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 482/975] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:57.114806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f53db-a4b5-4645-9e1b-d2a96895fbac None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.114915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.161971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.162702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.162984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.163255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.174424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.175873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:57.193609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:57.194642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:57.195679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:57.195679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:57.195679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:57.195679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:57.195679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 494/976] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:57.201525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.248021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e300293-5bc4-48db-9afb-0cd61a9b4958 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.255822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.285721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.286427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.286741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.287018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.287718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:57.288025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.289897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.294701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 483/977] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:57.300576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.337200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d92afa-a47f-4b34-bfd3-b703fd71d3b1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.359216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.394565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.395243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.395594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.395883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.396639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:57.396951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.401487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.416325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 495/978] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:57.421981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.430895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2334-2f84-4bb5-8bfd-ffd35c7b7561 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.468730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.469535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.469732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.470108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.470883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] RBAC: Adding query filter params (type=compute) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:57.471252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.472930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.478262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 484/979] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:57.484106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.521920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.549448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56b79d-2ba0-4894-9a56-2462d2ddb706 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.589345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.590157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.590456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.590722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.598453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.600206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.613745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629af8-77c3-4870-915a-e068ac85525b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:40:57.617112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:57.618055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 17:40:57.619107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:57.620924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 496/980] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:57.626588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.652085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.652739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.653060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.653326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.654059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] RBAC: Adding query filter params (interface=public, service_id=d45b303fca4749898495c42425b7eae3, region_id=RegionOne) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:57.654395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.655783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.661486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 485/981] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/endpoints?service_id=d45b303fca4749898495c42425b7eae3&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:57.663909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.667534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.749174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ccc9f-bad5-4411-9130-5b4fb42845c2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.780839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.785725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ec4b-433b-4143-86fa-d9023d61b6df None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:57.787709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.788389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.788672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.788953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.789802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:57.790119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.792135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.812329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 497/982] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:57.817970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:57.831648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:57.832474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:57.832758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:57.833030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:57.833725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:57.835137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:57.842065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 486/983] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:40:57.937636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:57.949069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1629e6-1948-4a55-adda-4e0a03d07378 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.002987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.002987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.002987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.002987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.083238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.084683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96771a4d-7e44-4d33-ba8d-d8083bce5820 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.199156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:58.216687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 498/984] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:40:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:40:58.222384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.338464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83692fe3-1391-474a-82cf-a6f7bd042ba6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.379579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.380372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.380653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.380926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.383885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e019-c7c2-4d8d-abd8-25c1fd37bdec None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.384198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e019-c7c2-4d8d-abd8-25c1fd37bdec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.384479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e019-c7c2-4d8d-abd8-25c1fd37bdec None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.386631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 499/985] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:58.391712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.391988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.392264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.394170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.395686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.410099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:58.411722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:40:58.415151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:58.423021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:58.424017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:40:58.424017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:40:58.424017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:40:58.424017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:58.424017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 487/986] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:40:58.425712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.443232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:58.493185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 500/987] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:40:58.516878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.553771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6a181-b8c7-4a44-9b46-c8ec2a52b7d9 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.605452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.606111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.606420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.606699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.607426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:58.607737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.609180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.622009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 488/988] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:40:58.628364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.647686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce743c1-041e-42fd-ba29-f7ccc2774914 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.679317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:58.695370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.696420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.696763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.697031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.698208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.699699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:58.712510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:58.713663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:40:58.714648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 501/989] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:58.717557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:58.719982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.758841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba2810-61b7-4882-9ba0-c9835c5f8e70 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.805764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.805764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.806048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.806176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.813966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.815552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:40:58.830842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:40:58.832243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:40:58.833906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 489/990] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:40:58.837811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.865433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c369bc-7d86-495e-baeb-0698f7fb5291 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:58.904409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:58.905072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:58.905350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:58.905627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:58.906325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:58.906835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:58.908211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:58.914123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 502/991] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:58.919722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:58.955232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8a7a1-61b8-4753-872e-b912a97dcb32 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:59.001069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:59.008798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:59.009628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.010496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.010775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.011566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:59.011875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:59.013362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:59.016502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:59.029884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 490/992] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:59.035621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:59.066118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4053-0291-4717-bfb4-b2b505094c58 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:59.108010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:59.109004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.109413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.109835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.110729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:59.111160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:59.112649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:59.117897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 503/993] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:40:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:59.124808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:59.164455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b05be6-b012-43b2-82a8-f4f43ad46034 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:59.207380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:59.208045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.208328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.208596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.209428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] RBAC: Adding query filter params (role.id=2ff5dd876493418c83374896107ec3b1, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=9425457963714156a39a1eced4c8f85f) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:59.209856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:59.211339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:59.218021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 491/994] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:40:59 2022] GET /identity/v3/role_assignments?user.id=9425457963714156a39a1eced4c8f85f&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=2ff5dd876493418c83374896107ec3b1 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:59.254390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000879c8-c3ab-4c0c-b422-770518e45928 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:59.271831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:59.307901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:59.308584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.308873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.309153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.309969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] RBAC: Adding query filter params (interface=admin, service_id=457c4b97694d41e49ab2ac635d518811, region_id=RegionOne) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:40:59.310289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:59.311738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9a9c-213d-4893-a20b-8cbf2192ac4e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:59.317990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 504/995] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Jun 30 17:40:59 2022] GET /identity/v3/endpoints?service_id=457c4b97694d41e49ab2ac635d518811&interface=admin®ion_id=RegionOne => generated 193 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:40:59.324242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:40:59.396596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:40:59.450723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901edddd-51af-4b3e-87f6-b78e14da656d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:40:59.491346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:40:59.492008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.492289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.492563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.493236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:40:59.494646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bed23-582d-4dac-b864-8f21d4655e4d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:40:59.507221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 492/996] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:40:59 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:40:59.942017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf476240-03bd-4216-bcf5-30acbac364e1 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.942573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf476240-03bd-4216-bcf5-30acbac364e1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.942653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf476240-03bd-4216-bcf5-30acbac364e1 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.944167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 505/997] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:40:59.950183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:40:59.950680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:40:59.950796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:40:59.976052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:40:59.977293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:00.054685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 493/998] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:40:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:00.077948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf476240-03bd-4216-bcf5-30acbac364e1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:00.202252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf476240-03bd-4216-bcf5-30acbac364e1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:00.242444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:00.243243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:00.243523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:00.243791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:00.244983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:00.246356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:00.258205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:00.259320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 17:41:00.260848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:00.262909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 506/999] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:41:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:00.266288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:00.328823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:00.378972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8991a3a-8c2b-46bb-a054-db0af68dd71e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:00.420184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:00.420851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:00.421132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:00.421409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:00.422133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:00.422496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:00.423868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:00.434360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 494/1000] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:41:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:00.439438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:00.487744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:00.551572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c5cd-1bf8-45b6-9070-512157552c3b None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:00.587007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:00.587683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:00.588502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:00.588774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:00.589481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:00.589803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:00.591172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62a761-3ea4-4228-8d97-b40215981331 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:00.596806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 507/1001] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jun 30 17:41:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:00.602246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:00.721557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4f637-4abb-4e8a-80a4-86a520726f00 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:00.738537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:00.758315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:00.759006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:00.759285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:00.760106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:00.760853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:00.762215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfc3fd-cfe2-4de3-b1de-512523cc93c8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:00.789790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 495/1002] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:41:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:01.343964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4d67-b213-442d-b429-ebbab375eb09 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.344531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4d67-b213-442d-b429-ebbab375eb09 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.344531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df4d67-b213-442d-b429-ebbab375eb09 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.346109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 508/1003] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:01.351353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6823e-77c1-4081-92bf-ba77dfc79d74 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.351949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6823e-77c1-4081-92bf-ba77dfc79d74 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.351949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6823e-77c1-4081-92bf-ba77dfc79d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.369857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0fb6823e-77c1-4081-92bf-ba77dfc79d74 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:01.371099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0fb6823e-77c1-4081-92bf-ba77dfc79d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:01.471695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 496/1004] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:01 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:01.495443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df4d67-b213-442d-b429-ebbab375eb09 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:01.577530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5608a-8e4f-412b-9a26-2946a0e7769c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.577856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5608a-8e4f-412b-9a26-2946a0e7769c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.578097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5608a-8e4f-412b-9a26-2946a0e7769c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.579511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 497/1005] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:01.584742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.585026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.585307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.602241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:01.603963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:01.618267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df4d67-b213-442d-b429-ebbab375eb09 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:01.626403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:01.663580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:01.664519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.664803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.665077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] PATH_INFO: `/v3/services/image` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.665798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:01.667729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:01.680054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:01.681354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:01.682771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 509/1006] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 17:41:01 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:01.687256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:01.701118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 498/1007] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:01 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:01.724263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:01.755028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:01.811885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5267c-1e81-4a07-bd51-7dbb2ff01bea None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:01.851562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68cb33-1279-4bb8-9161-8195a98e4c13 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:01.856771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:01.857384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.857666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.857941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.858687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] RBAC: Adding query filter params (name=image) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:01.858995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:01.860834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:01.867247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 510/1008] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:01.878561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:01.888787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:01.889610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:01.889903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:01.890168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:01.890862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:01.892244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:01.912361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:01.913439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:01.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 499/1009] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jun 30 17:41:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:41:01.920196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:01.923210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:02.018071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8d0c1-bde3-4186-b445-2e2f36a55668 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:02.047191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:02.056554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd94ea-eee5-4e0f-81e1-ea2b70dff743 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:02.063261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:02.063978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.064266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.064548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:02.065259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] RBAC: Adding query filter params (type=image) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:02.065566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:02.073190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:02.078689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 511/1010] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:02.101182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:02.101471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.101556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.101792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:02.102539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:02.102856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:02.104326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:02.110434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 500/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:41:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:02.116047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:02.228078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:02.233431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673b613-9b82-427b-a17e-79305a3e1b92 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:02.275932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:02.276599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.276881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.277157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:02.277864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:02.278170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:02.279576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059ea5d-5211-4914-a0d2-d21f1e1de3d9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:02.285042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 512/1012] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jun 30 17:41:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:02.292099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:02.412723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a8109-735d-4777-89d8-3b64b52f18b3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:02.455346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:02.456007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.456290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.456569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:02.457341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:02.458747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094811d3-15b1-4de4-8f7f-634932f513b6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:02.482362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 501/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:41:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:02.973868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba54ec-ca16-4d97-85dd-05fda58e0d0c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.974437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba54ec-ca16-4d97-85dd-05fda58e0d0c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.974437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba54ec-ca16-4d97-85dd-05fda58e0d0c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:02.975808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 513/1014] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:02.981933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:02.982294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:02.982678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:03.000519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:03.001797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:03.013884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:03.094529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 502/1015] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:03.116683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba54ec-ca16-4d97-85dd-05fda58e0d0c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:03.237813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba54ec-ca16-4d97-85dd-05fda58e0d0c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:03.276904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:03.277715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:03.277996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:03.278269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:03.296562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:03.307042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:03.308009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:03.308898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:03.310774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 514/1016] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:41:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:03.315105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:03.428143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1cf42-c012-4db2-8d89-42b9d4f82d8d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:03.465173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:03.465836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:03.466117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:03.466412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:03.467155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:03.467458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:03.468814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:03.482244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 503/1017] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:41:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:03.486429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:03.511894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:03.593937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db5a1b-a0f3-4819-ac74-119a68962898 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:03.629068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:03.629884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:03.630169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:03.630513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:03.643239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:03.644717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:03.663664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:03.664647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:03.671411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:03.671411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:03.671411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:03.671411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:03.671411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 515/1018] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:03.673257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:03.677299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:03.786290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14896c-1f76-4855-b6ae-097386d872eb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:03.826669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:03.827338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:03.827625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:03.827906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:03.828629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:03.828939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:03.830371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:03.845559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 504/1019] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:03.854374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:03.934033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:03.967601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac610-b048-499f-a0d1-9c2341e5e7cc None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.008932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.010290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.010609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.010882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.018764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.020201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-6536b094-fe9c-4fc7-b53e-ed1df869db23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:41:04.034302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:04.035321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:04.036360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:04.038669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 516/1020] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:41:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:04.041436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.108497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.150346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4e930-4d47-44c3-b888-46f4c3f48ea7 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.186827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.187504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.187785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.188062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.188885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:04.189192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.190645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.193296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f6589-7630-489e-a81e-16d35c08593e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.193593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f6589-7630-489e-a81e-16d35c08593e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.193877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f6589-7630-489e-a81e-16d35c08593e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.195310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 517/1021] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:04.201380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e8301-afef-4841-926e-d54fe4549002 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.201671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e8301-afef-4841-926e-d54fe4549002 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.201951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e8301-afef-4841-926e-d54fe4549002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.212033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 505/1022] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:04.217707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.222670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-944e8301-afef-4841-926e-d54fe4549002 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:04.223867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-944e8301-afef-4841-926e-d54fe4549002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:04.293883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 518/1023] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:04.318087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e8301-afef-4841-926e-d54fe4549002 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.323582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd05c3-c243-410d-af3b-93123a77dd85 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.348117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.362624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.363278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.363564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.363840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.394997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.437146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e8301-afef-4841-926e-d54fe4549002 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.439412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.440894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387b746-bf7d-4bc7-805f-e5206f9da52d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.477086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea874c12-714e-478d-877e-699e8881762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.477752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea874c12-714e-478d-877e-699e8881762a None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.478591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea874c12-714e-478d-877e-699e8881762a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.478881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea874c12-714e-478d-877e-699e8881762a None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.479607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea874c12-714e-478d-877e-699e8881762a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.480980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea874c12-714e-478d-877e-699e8881762a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.506364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 519/1024] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:41:04 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:41:04.557887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 506/1025] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:41:04.559628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea874c12-714e-478d-877e-699e8881762a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.563881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f34-2ca0-42f1-bec9-c2ee9fa11222 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.564171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f34-2ca0-42f1-bec9-c2ee9fa11222 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.564452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c5f34-2ca0-42f1-bec9-c2ee9fa11222 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.566264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 507/1026] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:04.572594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.572889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.573172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.591642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:04.592912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:04.613693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.679517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 508/1027] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:04.686552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea874c12-714e-478d-877e-699e8881762a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.705558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.718844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.732611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.733410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.733687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.733954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.747171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.748643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:04.769905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:04.771122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:04.772056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:04.772056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:04.772056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:04.772056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:04.772056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 520/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:04.778101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.822952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac51e04-7238-46cc-8d90-81e9a3b677b1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.867369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.868172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.868476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.868762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.870225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.871716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.873578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 509/1029] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:04.879306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.886598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0353d-dd53-47fd-a123-407c1dd20f97 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:04.897878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.927487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:04.928160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:04.928445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:04.928781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:04.929504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:04.929860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:04.931517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:04.949120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 521/1030] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:04.950201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:04.990756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:04.995839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f8f33-fe14-44a8-83ff-62608be0d1c8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:05.034146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:05.034838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:05.035305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:05.035580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:05.036283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:05.038212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:05.064671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823059-4e1a-4f72-9e86-cd1f7f57b12c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:05.066574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 510/1031] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:41:04 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:41:05.105291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:05.106021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:05.106303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:05.106610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:05.114505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:05.115994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:41:05.131748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:05.132742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:05.133669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 522/1032] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:05.140674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:05.220350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:05.258353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a71c5-3d30-4ef7-9ba7-6ae09dd7ac27 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:05.294158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:05.294867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:05.295148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:05.295424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:05.296217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:05.296526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:05.297955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea0a-10a8-436c-a578-c70fc9624750 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:05.311090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 511/1033] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:05.317347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:05.415667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:05.422877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880206c5-120e-4b8f-aeaf-26d22bc0e165 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:05.458627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:05.459284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:05.459570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:05.459845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:05.460678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] RBAC: Adding query filter params (role.id=2ff5dd876493418c83374896107ec3b1, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:05.460990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:05.462465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc992cfb-34d9-4bf0-8ddd-354646eebdcd None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:05.474437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 523/1034] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:05 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=2ff5dd876493418c83374896107ec3b1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:06.611896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4c78-1582-448a-8551-b3e5902c0d5f None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:06.612420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4c78-1582-448a-8551-b3e5902c0d5f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:06.612500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c4c78-1582-448a-8551-b3e5902c0d5f None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:06.613982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 512/1035] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:06.619904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:06.620343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:06.620521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:06.637173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:06.638397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:06.715785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 524/1036] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:06.739282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c4c78-1582-448a-8551-b3e5902c0d5f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:06.849283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c4c78-1582-448a-8551-b3e5902c0d5f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:06.877626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:06.886700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:06.887497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:06.887785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:06.888064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] PATH_INFO: `/v3/services/image` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:06.888719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:06.890094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:06.902724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:06.903774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:06.904638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 513/1037] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 17:41:06 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:06.911700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.014323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:07.026112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df18a1-e890-4cf6-8029-7def1504cd96 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.061151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.061828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.062115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.062420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.063128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] RBAC: Adding query filter params (name=image) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:07.063469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.064804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dcd94d-3b3c-4754-95c5-d9bcfd330a51 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.074750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 525/1038] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:07.076075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.181031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06031347-c6ed-4fde-ac94-9c9893fa985b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.185535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d43b-003f-4c31-8110-d57d57dab11b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.185822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d43b-003f-4c31-8110-d57d57dab11b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.186094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d43b-003f-4c31-8110-d57d57dab11b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.187657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 526/1039] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:07.192711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.193000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.193282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.210819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:07.212105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:07.219620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.220717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.221213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.221700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.223014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] RBAC: Adding query filter params (type=image) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:07.223563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.226055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.240066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 514/1040] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:07.241378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.289940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 527/1041] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:07.300161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:07.314530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.352049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc612695-d23d-46d5-8b75-fd7a2937a644 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.402105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.403318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.403807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.404207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:07.404338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.405994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] RBAC: Adding query filter params (interface=public, service_id=742b303b34a94320936645fb887d6372, region_id=RegionOne) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:07.406540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.409127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.418209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 515/1042] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/endpoints?service_id=742b303b34a94320936645fb887d6372&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:07.423612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.427167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2687b-b8c9-4a5c-af82-1f3978814536 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.464718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.465436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.465724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.465999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.467231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.468673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.470345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 528/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:07.475917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.542696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030ca5c-b5d5-4308-bd6c-a747f0b0059a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.562895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:07.577844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.578533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.578817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.579095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.579770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.581139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd058c-80fc-4721-b781-6616adbd27db None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.588324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 516/1044] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:41:07.589649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ced8-5d22-4613-afba-e449b00a27a5 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.589934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ced8-5d22-4613-afba-e449b00a27a5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.590210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ced8-5d22-4613-afba-e449b00a27a5 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.590888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb80899-efe4-4dad-8342-c14dac0e60a2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.591693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 517/1045] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:07.597516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.597876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.598079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.619310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:07.620548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:07.632942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.634975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.635408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.635914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.636922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:07.638639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:07.663094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 529/1046] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:41:07 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:41:07.709303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 518/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:07.733679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:07.747417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:07.857360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-215f-44f2-8e63-3b7fa8e94acf None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:07.895638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:07.896461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:07.896741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:07.897013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:07.908292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:07.916587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:07.917607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:07.918765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 530/1048] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:07.923886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.037312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521b048-3459-43cf-910b-2a12767ce608 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.052903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:08.075883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.076556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.076835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.077111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.077853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:08.078158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.079627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.093035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 519/1049] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:41:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:08.094004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.164495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:08.201516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac673d8-b2fa-4acd-b3de-bb6ae1ef02d1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.237384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.238615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.239107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.239583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.251535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.253306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:08.271337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:08.272348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:08.273299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:08.273299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:08.273299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:08.273299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:08.279011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 531/1050] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:08.280283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.393716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0ed08-a993-41b0-8391-d5d6f730d7cc None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.432724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.433755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.434262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.434785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.435722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:08.436231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.437871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.451679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 520/1051] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:08.457116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.561249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6686b24-7f3f-48d5-a700-cb8217497075 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.582148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:08.598168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.598987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.599274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.599552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.608037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.609487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:41:08.623495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:08.624465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:08.625489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:08.629430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 532/1052] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:08.630869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.666786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:08.738612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cb628-cfc4-406d-9266-016be1be6295 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.776043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.776717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.776998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.777267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.778101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:08.778436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.779863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.797913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 521/1053] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:08.803484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:08.910061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5a122-352f-4240-917e-55d536c75165 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:08.944726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:08.945530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:08.946075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:08.946361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:08.960678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:08.962195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:08.980316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:08.981297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:08.982262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:08.982262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:08.982262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:08.982262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:08.988755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 533/1054] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:08.991467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:09.092645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:09.100484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524a351-296d-45ca-ae96-3cc53ee122bb None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:09.143768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:09.144424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.144708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.144980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.145702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:09.146010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:09.147527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:09.164388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 522/1055] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:09.165784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:09.200377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:09.272759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94a8b3-ff12-4abe-96c7-0b75e23d07ab None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:09.307011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:09.307809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.308090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.308360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.315591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:09.317039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:41:09.332545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:09.333519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:09.339245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 534/1056] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:09.341267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:09.454169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa6c2-be5e-4553-bbeb-6c07a08cc074 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:09.487934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:09.488592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.488873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.489146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.489924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:09.490231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:09.491690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d66f1-64b3-48b8-99cd-3f1e0cc37157 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:09.510419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 523/1057] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:09.511723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:09.608227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:09.618319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ff819-8f96-4931-9ed1-c99170e16f8d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:09.655174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:09.656065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.656345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.656619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/2ff5dd876493418c83374896107ec3b1` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.658842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=c064d58c15704009864fc43a1e630676, role_id=2ff5dd876493418c83374896107ec3b1)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:09.660264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600992d2-385d-4c89-ac81-272ba33a4c34 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:09.682369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 535/1058] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:41:09 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/c064d58c15704009864fc43a1e630676/roles/2ff5dd876493418c83374896107ec3b1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:41:09.704977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b699-804f-4802-99c0-6facb350466d None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.705495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b699-804f-4802-99c0-6facb350466d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.705495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852b699-804f-4802-99c0-6facb350466d None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.706861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 524/1059] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:09.712635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315718-50e7-42e1-b5cb-ece757f74bbd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.712955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315718-50e7-42e1-b5cb-ece757f74bbd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.713188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315718-50e7-42e1-b5cb-ece757f74bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.731715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6e315718-50e7-42e1-b5cb-ece757f74bbd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:09.733082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6e315718-50e7-42e1-b5cb-ece757f74bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:09.815371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 536/1060] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:09.839604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852b699-804f-4802-99c0-6facb350466d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:09.859038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:09.898548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5eda5-cb25-4c89-b477-a5a095fbe87a None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.899290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5eda5-cb25-4c89-b477-a5a095fbe87a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.900233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5eda5-cb25-4c89-b477-a5a095fbe87a None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.901763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 537/1061] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:09.906955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:09.907246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:09.907526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:09.924358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:09.925578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:09.960983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852b699-804f-4802-99c0-6facb350466d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.000501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.001302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.001586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.001869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] PATH_INFO: `/v3/services/image` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.002656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.004225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.015947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 538/1062] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:10.023123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:10.024190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 17:41:10.025144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:10.026570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 525/1063] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jun 30 17:41:09 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:10.030299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.040505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.046695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.128243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.152774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532ebe2-b2aa-475b-b54e-a0277f23ddd3 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.154082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06615a-925c-46f6-8fd7-d3cbc1ba99d8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.191250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.191838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.191980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.192188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.192460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.192635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.192922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.193169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:10.193267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.193471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.194868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.205691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:10.205906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 539/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:10.206762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:10.213593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:10.214693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:10.215553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 526/1065] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:10.220912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.296662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.323546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958580e3-3960-42d2-8faa-a3747d1e66a5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.327483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe52104-ad96-4a08-8f7a-be1acb76a755 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.360688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.361353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.361630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.361901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.362670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:10.362967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.363305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.363970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.364257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.364489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.364594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.365236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] RBAC: Adding query filter params (type=image) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:10.365545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.367194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.370425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.381735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 540/1066] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:10.381934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 527/1067] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:10.382053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.383159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.485055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca093b6a-8f64-4f0a-b402-8cba2c5e6d76 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.485371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59763169-1043-43ff-9bf0-9a81bafe1e7c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.521264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.522243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.522566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.522837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] PATH_INFO: `/v3/users/46047890a44b450b88e9a50926445983` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.524152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.526395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.526814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.527104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.527861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.529238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.534210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.538087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] RBAC: Authorizing `identity:get_user(user_id=46047890a44b450b88e9a50926445983)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.540932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.547682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 541/1068] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/users/46047890a44b450b88e9a50926445983 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:10.548750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.562360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 528/1069] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:41:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:10.669114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5342aa-5b85-446f-ba20-45a0f399a742 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:10.729897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:10.730601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:10.730882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:10.731153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:10.798393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:10.806626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:10.808085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:10.920470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 542/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:10.926538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:10.993052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:11.041060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c4836-c0f9-416b-97a6-26176060c56a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:11.076829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:11.077693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:11.077976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:11.078253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:11.079505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:11.080969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480f6f8-caa0-45fd-a951-9c40b60589b7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:11.082982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 529/1071] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:10 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:11.089377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:11.184997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:11.194215 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154e777-1d9b-4bc7-b87d-d13bda8e8bad None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:11.230340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:11.231014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:11.231300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:11.231580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:11.232410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=1b1cb63c9ccc446ba16d899d05763be0, user.id=46047890a44b450b88e9a50926445983) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:11.232726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:11.234188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e235c2-62cd-43ac-9fb2-397db53bb3cc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:11.246378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 543/1072] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:11 2022] GET /identity/v3/role_assignments?user.id=46047890a44b450b88e9a50926445983&scope.project.id=1b1cb63c9ccc446ba16d899d05763be0&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:11.774003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9c30-e337-4319-8548-72f0c1aab9c2 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:11.774431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9c30-e337-4319-8548-72f0c1aab9c2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:11.774626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9c30-e337-4319-8548-72f0c1aab9c2 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:11.776174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 530/1073] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:11.781367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:11.781662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:11.781927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:11.799068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:11.800256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:11.870079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 544/1074] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:11.893548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9c30-e337-4319-8548-72f0c1aab9c2 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.009499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9c30-e337-4319-8548-72f0c1aab9c2 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.035116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.047276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.048104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.048385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.048658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.060262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:12.068563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:12.069559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:12.070893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 531/1075] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jun 30 17:41:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:12.076123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.166797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.180295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a166171-8a4f-47de-8dd4-14b21641dbe6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.215271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.215939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.216218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.216493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.217230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:12.217534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.218933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:12.228276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 545/1076] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:12.233594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.339753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28353e7-d63e-42cd-bcd2-314635d587c1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.379206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.380077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.380362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.380634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.392420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.393901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:12.412707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:12.413708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:12.420409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:12.420409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:12.420409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:12.420409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:12.420409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 532/1077] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:41:12.420697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.529581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb70e74-0783-42ec-b790-54808c4dde24 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.562935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.563759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.564048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.564323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.565035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:12.565343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.566803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe248-c630-4c85-be50-9dbb79c2af68 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:12.579502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 546/1078] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:12.584995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.620534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.642888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47a11-a87a-4e58-9942-3205e367f377 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.643178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47a11-a87a-4e58-9942-3205e367f377 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.643458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd47a11-a87a-4e58-9942-3205e367f377 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.644934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 547/1079] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:41:12.649515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.650303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.650609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.667117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:12.668305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:12.671108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.688930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34518bcc-fa8c-4880-a33a-eee90d983447 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.724582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.725373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.725654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.725928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.733139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.734718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:12.746661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 548/1080] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:41:12.748912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:12.749843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 17:41:12.750830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:12.754713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 533/1081] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:12.755665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.762934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.860142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.866613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fada8a-f665-4cc9-8422-21a4e751569e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.867266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823c887-d48f-4ee6-94c2-d34c5b68693a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:12.903648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.903861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:12.904213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.904417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:12.904519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.904702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:12.904801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.904982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:12.905812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:12.906122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.907891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:12.920685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:12.937660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 549/1082] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:12.938874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:12.982651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:12.984124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.057799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37158fc-61ed-49ab-831b-e03da76f1ee2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.093470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.094112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.094395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.094667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.097096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.107059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 534/1083] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:41:13.112862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.170134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.173556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.175069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.220154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746be0c-d027-4777-875f-dc3503c6f716 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.256612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.257757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.258271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.258637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.270025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.271512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.288014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 550/1084] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:13.289373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:13.289760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:13.291151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:13.292061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:13.292061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:13.292061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:13.292061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:13.292061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 535/1085] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:13.297059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.348814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.390749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.394446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341718aa-9dc0-4430-84b3-7e2b0540de49 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.399419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caedae9-a9c9-404f-9777-a3fb05259ad1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.430067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.431405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.431695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.431965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.435851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.436513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.436801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.437077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.437796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:13.438106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.439510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1123-6063-4428-8819-8120912a1182 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.441959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.443418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.451077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 536/1086] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:13.452261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8615-dea8-460d-8849-c422b7181282 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.452559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8615-dea8-460d-8849-c422b7181282 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.452844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8615-dea8-460d-8849-c422b7181282 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.458686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 537/1087] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:13.460151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.460443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.460721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:13.462991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:13.463911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:13.464931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:41:13.464931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:13.464931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:41:13.464931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:13.464931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 551/1088] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:41:13.470498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.481759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:13.483048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:13.567563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 538/1089] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:13.592362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.596632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5715d-be8f-418a-978e-ce157dd286f4 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.637446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.638115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.638464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.638745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.639473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:13.640409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.640542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.641401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.654013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 552/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:13.660275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.675138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.710925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25bce98-0b88-413b-aea9-d8a6cfbfc0a7 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.749910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.750590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.750873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.751144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.762540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:13.770155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:13.771283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:13.772535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 539/1091] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:13.778022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.778145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645aa7b-8e5d-4206-9caa-30f8e15880bf None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.813394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.814234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.814548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.814828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.822588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.824071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:41:13.839463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:13.840524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:13.841625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 553/1092] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:41:13.846435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.875278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.879532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102abb-8aae-4084-8345-fbecf6d750a1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.919102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.919868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.920275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.920756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.920928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:13.921407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:13.921735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.924238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:13.934508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 540/1093] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:13.940203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:13.956940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fed128-6bf3-4946-991e-a531ff5cceb6 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:13.993398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:13.994113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:13.994424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:13.994695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:13.995487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:13.995790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:13.997228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:14.009293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 554/1094] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:14.015499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:14.042383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7a471-04b3-4c9d-8849-aaecdb590a7a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:14.077337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:14.078207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:14.078527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:14.078802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] PATH_INFO: `/v3/users/46047890a44b450b88e9a50926445983` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:14.080014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] RBAC: Authorizing `identity:get_user(user_id=46047890a44b450b88e9a50926445983)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:14.081454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:14.083141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 541/1095] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:13 2022] GET /identity/v3/users/46047890a44b450b88e9a50926445983 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:14.091931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:14.119219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3946c1f-8797-4054-b27b-fb9226d31417 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:14.128350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:14.150491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:14.157667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:14.158311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:14.158729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:14.159012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:14.159830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] RBAC: Adding query filter params (role.id=2ff5dd876493418c83374896107ec3b1, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=c064d58c15704009864fc43a1e630676) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:14.160141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:14.161567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:14.169761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 555/1096] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:14 2022] GET /identity/v3/role_assignments?user.id=c064d58c15704009864fc43a1e630676&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=2ff5dd876493418c83374896107ec3b1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:14.205110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ba1-78bd-44b2-a5bd-945d570798a1 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:14.246447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:14.247255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:14.247537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:14.247811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:14.249076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:14.250574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd9f8f-ea1f-4412-b2fa-1f3f23fe56a9 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:14.252371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 542/1097] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:14 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:14.259170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:14.370147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568f228-9044-4ce5-92ec-511f24bf1897 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:14.413703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:14.414638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:14.415024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:14.415293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0/users/46047890a44b450b88e9a50926445983/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:14.417282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] RBAC: Authorizing `identity:create_grant(project_id=1b1cb63c9ccc446ba16d899d05763be0, user_id=46047890a44b450b88e9a50926445983, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:14.418736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd5734-1356-493a-98e3-0610a742292d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:14.433517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 556/1098] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:41:14 2022] PUT /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0/users/46047890a44b450b88e9a50926445983/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:41:16.463241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690c93-3b26-466d-9823-bb684201b764 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:16.463803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690c93-3b26-466d-9823-bb684201b764 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:16.463803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60690c93-3b26-466d-9823-bb684201b764 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:16.465267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 543/1099] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:16.470502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:16.470872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:16.471066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:16.487990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:16.489167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:16.536888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:16.555267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 557/1100] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:16.579543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60690c93-3b26-466d-9823-bb684201b764 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:16.679313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:16.686583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60690c93-3b26-466d-9823-bb684201b764 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:16.724477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:16.725195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:16.725480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:16.725779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:16.736464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:16.744036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:16.745231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:16.746107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 544/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:16.751315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:16.848450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86776587-09e8-4cee-b9a1-7b4219e7fab1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:16.882135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:16.883124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:16.883559 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:16.883878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:16.884772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:16.885101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:16.886647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:16.894803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 558/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:16.901303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:17.000160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e4a7c-a8f7-47dd-b997-d0c85b69ec59 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:17.033348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:17.034217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:17.034533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:17.034810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] PATH_INFO: `/v3/users/46047890a44b450b88e9a50926445983` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:17.035992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] RBAC: Authorizing `identity:get_user(user_id=46047890a44b450b88e9a50926445983)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:17.037438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:17.039111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 545/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:16 2022] GET /identity/v3/users/46047890a44b450b88e9a50926445983 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:17.044966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:17.090709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:17.145577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e0e84-34f6-4f43-90f8-afc1325b94eb None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:17.179004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:17.179637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:17.179918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:17.180189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:17.245643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:17.247032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:17.291096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:17.344811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 559/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:17.350753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:17.377332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:17.455728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8c78-4289-410c-8808-ee99e0eab12d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:17.488374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:17.489211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:17.489488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:17.489756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:17.490963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:17.492389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff3b39-5587-42f7-8025-a4123f0641fb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:17.494641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 546/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:17 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:17.500797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:17.599682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975788-9e09-4e47-b6f6-61718c67691c None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:17.641509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:17.642183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:17.643584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:17.643584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:17.643584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=1b1cb63c9ccc446ba16d899d05763be0, user.id=46047890a44b450b88e9a50926445983) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:17.644279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:17.645203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d996c8b-6c5b-4a6a-9c62-5f5d8db7b060 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:17.651074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 560/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:17 2022] GET /identity/v3/role_assignments?user.id=46047890a44b450b88e9a50926445983&scope.project.id=1b1cb63c9ccc446ba16d899d05763be0&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:19.719419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824848-3a72-4439-8342-75a1eee1db67 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:19.720010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824848-3a72-4439-8342-75a1eee1db67 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:19.720010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa824848-3a72-4439-8342-75a1eee1db67 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:19.721365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 547/1107] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:19.726885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:19.727367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:19.728991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:19.745526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:19.746756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:19.749549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:19.815264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 561/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:19.841424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa824848-3a72-4439-8342-75a1eee1db67 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:19.941256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:19.951850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa824848-3a72-4439-8342-75a1eee1db67 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:19.985879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:19.986625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:19.986912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:19.987190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:19.988373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:19.989810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57ac38-7710-4f73-9fc3-425837a6c471 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:19.991462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 548/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:19.997821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:20.097160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef9233-23d7-43e7-915c-59d36d845f9a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:20.131651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:20.132318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:20.132602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:20.132875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:20.133625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:20.135077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751b4a5-4e2c-4f58-a26f-222eaae249a4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:20.144979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:20.161170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 562/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:41:19 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:41:22.239397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a827-4d53-426d-b808-f355731b9128 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:22.239951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a827-4d53-426d-b808-f355731b9128 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:22.240166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a827-4d53-426d-b808-f355731b9128 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:22.241797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 549/1111] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:22.247037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:22.247462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:22.247584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:22.264595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:22.265804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:22.331556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 563/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:22.355418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a827-4d53-426d-b808-f355731b9128 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:22.458415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a827-4d53-426d-b808-f355731b9128 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:22.492126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:22.492904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:22.493187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:22.493463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] PATH_INFO: `/v3/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:22.494957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] RBAC: Authorizing `identity:get_role(role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:22.496323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:22.497724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 550/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:22 2022] GET /identity/v3/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:41:22.503646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:22.609278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bedcd-3dbe-4b65-b500-18b30419e340 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:22.636133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:22.644931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:22.645813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:22.646095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:22.646396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] PATH_INFO: `/v3/users/df97663c0140401c9aed49b0db20de89` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:22.661805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] RBAC: Authorizing `identity:get_user(user_id=df97663c0140401c9aed49b0db20de89)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:22.663320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:22.665189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 564/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:22 2022] GET /identity/v3/users/df97663c0140401c9aed49b0db20de89 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:22.671338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:22.702951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:22.774128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d71b-e3d1-4e8a-9bad-415c55b363a8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:22.806677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:22.807310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:22.807589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:22.807859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:22.873064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:22.874527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:22.900430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:22.972091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 551/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:22.977909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:23.077404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1efd27-03cd-417a-8edf-ee59c4988f8a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:23.110915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:23.111759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:23.112037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:23.112306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:23.113462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:23.114923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f3c53-e039-4f6b-8bbe-86a749c3a27e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:23.116628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 565/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:22 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:23.123336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:23.219695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b7ad6-1873-48d3-a8b7-72b27995bc03 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:23.245932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:23.254314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:23.254991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:23.255274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:23.255546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:23.256358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=1b1cb63c9ccc446ba16d899d05763be0, user.id=df97663c0140401c9aed49b0db20de89) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:23.256665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:23.258127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065d4ad-8b0b-45d3-a1f8-9099ffadc606 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:23.263755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 552/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:23 2022] GET /identity/v3/role_assignments?user.id=df97663c0140401c9aed49b0db20de89&scope.project.id=1b1cb63c9ccc446ba16d899d05763be0&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:25.294691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de39fe-7088-4b7e-9f47-44b956674403 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.295220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de39fe-7088-4b7e-9f47-44b956674403 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.295220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de39fe-7088-4b7e-9f47-44b956674403 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.296539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 566/1118] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:25.301979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.302432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.302575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.319372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:25.320838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:25.385697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 553/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:25.408429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de39fe-7088-4b7e-9f47-44b956674403 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:25.463132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:25.512247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de39fe-7088-4b7e-9f47-44b956674403 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:25.549908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:25.551841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.552124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.552400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] PATH_INFO: `/v3/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.554844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] RBAC: Authorizing `identity:get_role(role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:25.557532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:25.558952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 567/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:25 2022] GET /identity/v3/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:25.564723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:25.660839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9df42-ed40-4456-989b-69ba1fbf89e9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:25.683486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:25.694813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:25.695983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.696266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.696542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] PATH_INFO: `/v3/users/df97663c0140401c9aed49b0db20de89` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.697727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] RBAC: Authorizing `identity:get_user(user_id=df97663c0140401c9aed49b0db20de89)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:25.699195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:25.700840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 554/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:25 2022] GET /identity/v3/users/df97663c0140401c9aed49b0db20de89 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:25.706715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:25.809537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f7b66-d9d4-4993-9f36-0d04674b4d74 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:25.829410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:25.844554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:25.845407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.845690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.845962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.847168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:25.848598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7b7b0-3bcc-41bb-aee0-d7096d840f18 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:25.850305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 568/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:25 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:25.857196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:25.953612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616cc03b-e736-4962-bfad-7f2fbf29593c None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:25.987965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:25.988851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:25.989134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:25.989407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0/users/df97663c0140401c9aed49b0db20de89/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:25.991663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b1cb63c9ccc446ba16d899d05763be0, user_id=df97663c0140401c9aed49b0db20de89, role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:25.993133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013ddd6-cc1a-46e9-9b30-a3e7fe684381 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:26.005870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 555/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:41:25 2022] PUT /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0/users/df97663c0140401c9aed49b0db20de89/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:41:27.998971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34645c6-bbc2-4397-8203-0672fe614b3c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:27.999423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34645c6-bbc2-4397-8203-0672fe614b3c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:27.999525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34645c6-bbc2-4397-8203-0672fe614b3c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.000882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 569/1124] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:28.006155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.006593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.006765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.023977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:28.025280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:28.072108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:28.093603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 556/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:28.117086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34645c6-bbc2-4397-8203-0672fe614b3c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:28.217620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34645c6-bbc2-4397-8203-0672fe614b3c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:28.251355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:28.252102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.252378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.252646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] PATH_INFO: `/v3/roles/d94f7f81626c4a6abc27b575874e9e25` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.254109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] RBAC: Authorizing `identity:get_role(role_id=d94f7f81626c4a6abc27b575874e9e25)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:28.255495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:28.256871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 570/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:28 2022] GET /identity/v3/roles/d94f7f81626c4a6abc27b575874e9e25 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:28.263248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:28.362727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63d34c-643a-4c18-874d-a91d82d5d54b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:28.396556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:28.397418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.397700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.397974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] PATH_INFO: `/v3/users/df97663c0140401c9aed49b0db20de89` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.399200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] RBAC: Authorizing `identity:get_user(user_id=df97663c0140401c9aed49b0db20de89)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:28.400633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:28.402261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 557/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:28 2022] GET /identity/v3/users/df97663c0140401c9aed49b0db20de89 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:28.408019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:28.463493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:28.507768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054deb8-f085-4c2d-bc59-b47786d63233 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:28.540464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:28.541105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.541383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.541652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.606932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:28.608373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:28.660958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:28.705007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 571/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:28.711378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:28.759130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:28.814428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc22c8-efc9-4e5e-a6e2-af32dfa5ec5d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:28.847562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:28.848413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.848692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.848965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] PATH_INFO: `/v3/projects/1b1cb63c9ccc446ba16d899d05763be0` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.850133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] RBAC: Authorizing `identity:get_project(project_id=1b1cb63c9ccc446ba16d899d05763be0)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:28.852225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee43f5-0bdb-428b-afe2-f95fe561192e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:28.853933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 558/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:28 2022] GET /identity/v3/projects/1b1cb63c9ccc446ba16d899d05763be0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:28.860020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:28.956558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e561f26-59b9-4728-af3b-522bb23e54c8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:28.991697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:28.992348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:28.992624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:28.992893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:28.993703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] RBAC: Adding query filter params (role.id=d94f7f81626c4a6abc27b575874e9e25, scope.project.id=1b1cb63c9ccc446ba16d899d05763be0, user.id=df97663c0140401c9aed49b0db20de89) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:28.994009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:28.995459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6963e2-46fd-496f-8355-af8769d0d6ac None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:29.001307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 572/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:28 2022] GET /identity/v3/role_assignments?user.id=df97663c0140401c9aed49b0db20de89&scope.project.id=1b1cb63c9ccc446ba16d899d05763be0&role.id=d94f7f81626c4a6abc27b575874e9e25 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:31.067104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cd04-98cc-45a4-aa79-142f5026adf6 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:31.067618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cd04-98cc-45a4-aa79-142f5026adf6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:31.067711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cd04-98cc-45a4-aa79-142f5026adf6 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:31.069008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 559/1131] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:31.074448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:31.074823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:31.074999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:31.091609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:31.092767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:31.106737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:31.157109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 573/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:31.180482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9cd04-98cc-45a4-aa79-142f5026adf6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:31.277906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9cd04-98cc-45a4-aa79-142f5026adf6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:31.293484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:31.310672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:31.311446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:31.311723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:31.311991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:31.313139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:31.314594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2de77-7fcf-45f4-965b-1651eb4f9dc4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:31.316186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 560/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:31.322844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:31.417535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98cc07-9c49-4775-be20-181f6493510f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:31.450346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:31.451000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:31.451285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:31.451557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:31.452264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:31.453690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60170b-4418-480e-bd41-a71bc07b7f4e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:31.478885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 574/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:41:31 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:41:33.523917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7beece-a220-415e-ba41-63f965c02fcf None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:33.524447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7beece-a220-415e-ba41-63f965c02fcf None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:33.524447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7beece-a220-415e-ba41-63f965c02fcf None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:33.525817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 561/1135] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:33.531088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:33.531527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:33.531652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:33.536656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:33.550225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:33.551457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:33.619508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 575/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:33.646848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7beece-a220-415e-ba41-63f965c02fcf None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:33.753782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7beece-a220-415e-ba41-63f965c02fcf None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:33.789522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:33.790233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:33.790543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:33.790816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:33.792016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:33.793458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec40ed-28a0-47c8-946b-d4a942b63c64 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:33.795138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 562/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:41:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:33.800971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:33.900020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ab2ad-90a4-4f29-8673-b8f6daad58a1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:33.929114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:33.934226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:33.934996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:33.935278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:33.935549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:33.936260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:33.937680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ea4ce-4400-40cc-9b8a-fd8d7fa60686 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:33.958854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 576/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:41:33 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:36.043458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1530-a531-49a6-a281-8da3c8d0d02b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.043928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1530-a531-49a6-a281-8da3c8d0d02b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.044020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1530-a531-49a6-a281-8da3c8d0d02b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.045351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 563/1139] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:36.050594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.050991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.051162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.067437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:36.068630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:36.138478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 577/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:36.162594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1530-a531-49a6-a281-8da3c8d0d02b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:36.215346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:36.264584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1530-a531-49a6-a281-8da3c8d0d02b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:36.298108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:36.298904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.299186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.299452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.309810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:36.317293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:36.318507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:36.319949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 564/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:36.325395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:36.426813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbadcd-b2be-4e5a-a561-008592a4ceb7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:36.447663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:36.467776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:36.468435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.468715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.468987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.469722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:36.470026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:36.471411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:36.479528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 578/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:36.485360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:36.585635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e582e0-d593-474d-9492-fe4873467897 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:36.598612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:36.620403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:36.621266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.621546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.621814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] PATH_INFO: `/v3/users/fec6326157a043e09236360eb175d76c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.635430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] RBAC: Authorizing `identity:get_user(user_id=fec6326157a043e09236360eb175d76c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:36.637289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:36.639203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 565/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:36 2022] GET /identity/v3/users/fec6326157a043e09236360eb175d76c => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:36.645622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:36.758260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f9ce7-f435-45b5-9ff0-4cb1cb51c867 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:36.791611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:36.792254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:36.792537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:36.792812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:36.844811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:36.867431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:36.868873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:36.973524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 579/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:36.980230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:37.101787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55905296-93b3-4d0d-a011-83f1d655311f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:37.136131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:37.136988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:37.137273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:37.137550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] PATH_INFO: `/v3/projects/82cea0f40d654bb18fddf6b0854a3adc` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:37.138744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] RBAC: Authorizing `identity:get_project(project_id=82cea0f40d654bb18fddf6b0854a3adc)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:37.140173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1a79-3141-46f7-b2ee-34bfe7340f49 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:37.141868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 566/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:36 2022] GET /identity/v3/projects/82cea0f40d654bb18fddf6b0854a3adc => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:37.148804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:37.225009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:37.244179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8ea1f-ee5a-4aed-af21-b345e65f9792 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:37.276883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:37.277515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:37.277795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:37.278067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:37.278909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=82cea0f40d654bb18fddf6b0854a3adc, user.id=fec6326157a043e09236360eb175d76c) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:37.279216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:37.280619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9212336-24e7-4a2a-a45d-ba14fd9408f8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:37.285914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 580/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:37 2022] GET /identity/v3/role_assignments?user.id=fec6326157a043e09236360eb175d76c&scope.project.id=82cea0f40d654bb18fddf6b0854a3adc&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:39.326017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa52d-2540-4d22-86d8-ef488cc70a9f None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:39.326593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa52d-2540-4d22-86d8-ef488cc70a9f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:39.326593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa52d-2540-4d22-86d8-ef488cc70a9f None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:39.327984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 567/1147] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:39.333292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:39.333755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:39.333858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:39.350163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:39.351404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:39.421082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 581/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:39.444213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa52d-2540-4d22-86d8-ef488cc70a9f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:39.484594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:39.546794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa52d-2540-4d22-86d8-ef488cc70a9f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:39.580758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:39.581456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:39.581735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:39.582004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:39.592981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:39.601551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:39.602737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:39.603640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 568/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:39.608994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:39.683886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:39.710405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126418c3-024a-4c27-99b5-c32c78a04338 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:39.744401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:39.745055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:39.745334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:39.745602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:39.746346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:39.746659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:39.748006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:39.756060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 582/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:39.761842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:39.854928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:39.858782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef7733-648d-4a3a-bcd9-c0b4f55be40f None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:39.891858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:39.892720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:39.893002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:39.893273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] PATH_INFO: `/v3/users/fec6326157a043e09236360eb175d76c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:39.894470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] RBAC: Authorizing `identity:get_user(user_id=fec6326157a043e09236360eb175d76c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:39.895907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:39.898047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 569/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:39 2022] GET /identity/v3/users/fec6326157a043e09236360eb175d76c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:39.904126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:40.015815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd4728-d3ba-40fe-897f-7612a9c68c68 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:40.049397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:40.050252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:40.050576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:40.050846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] PATH_INFO: `/v3/projects/82cea0f40d654bb18fddf6b0854a3adc` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:40.052042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=82cea0f40d654bb18fddf6b0854a3adc)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:40.053464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d2f68-14e7-426d-83fa-75fa2220a9f4 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:40.055239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 583/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:39 2022] GET /identity/v3/projects/82cea0f40d654bb18fddf6b0854a3adc => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:40.061323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:40.160637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83467c1-29b1-4b6d-81ab-ed5066459d00 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:40.193023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:40.193924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:40.194203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:40.194801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] PATH_INFO: `/v3/projects/82cea0f40d654bb18fddf6b0854a3adc/users/fec6326157a043e09236360eb175d76c/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:40.196743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] RBAC: Authorizing `identity:create_grant(project_id=82cea0f40d654bb18fddf6b0854a3adc, user_id=fec6326157a043e09236360eb175d76c, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:40.198166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510de44-499a-4dd2-b85c-1a17855da9fb None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:40.211906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 570/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:41:40 2022] PUT /identity/v3/projects/82cea0f40d654bb18fddf6b0854a3adc/users/fec6326157a043e09236360eb175d76c/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:41:42.251409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64829-d7cf-43ee-abd7-d83ba2b8e24c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.251909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64829-d7cf-43ee-abd7-d83ba2b8e24c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.251909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64829-d7cf-43ee-abd7-d83ba2b8e24c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:42.253241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 584/1154] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:42.258509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.258951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.259056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:42.276425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:42.277652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:42.299326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:42.345673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 571/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:42.372492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64829-d7cf-43ee-abd7-d83ba2b8e24c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:42.376304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:42.472948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64829-d7cf-43ee-abd7-d83ba2b8e24c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:42.505954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:42.506685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.507565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.507837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:42.517587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:42.525707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:42.526873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:42.527939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 585/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:42.532995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:42.629385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d41e3-6225-4bb3-b95c-777e5d3757e3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:42.662703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:42.663390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.663655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.663930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:42.664669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:42.664975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:42.666366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:42.674547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 572/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:42.680369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:42.742640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:42.781228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368776da-77ce-493c-b7b1-b10d7fb6b395 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:42.813157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:42.814013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.814294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.814595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] PATH_INFO: `/v3/users/fec6326157a043e09236360eb175d76c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:42.815756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] RBAC: Authorizing `identity:get_user(user_id=fec6326157a043e09236360eb175d76c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:42.817176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:42.818845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 586/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:42 2022] GET /identity/v3/users/fec6326157a043e09236360eb175d76c => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:42.825381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:42.845376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:42.928027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc50950-1112-42bd-ad1a-2b72e3dabb10 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:42.960885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:42.961525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:42.961802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:42.962068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:43.027002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:43.028442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:43.045986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:43.124052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 573/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:43.129949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:43.227720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e78488-8f93-4fad-be03-6539cbce371e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:43.255446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:43.261348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:43.262214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:43.262527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:43.262808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] PATH_INFO: `/v3/projects/82cea0f40d654bb18fddf6b0854a3adc` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:43.264001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] RBAC: Authorizing `identity:get_project(project_id=82cea0f40d654bb18fddf6b0854a3adc)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:43.265427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85331071-0f81-47f1-987e-71c991e5c038 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:43.267167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 587/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:43 2022] GET /identity/v3/projects/82cea0f40d654bb18fddf6b0854a3adc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:43.273767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:43.371120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68724d9-db4c-4073-8126-b3376082ffe5 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:43.388452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:43.406322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:43.407013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:43.407292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:43.407562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:43.409046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=82cea0f40d654bb18fddf6b0854a3adc, user.id=fec6326157a043e09236360eb175d76c) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:43.409380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:43.410934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b5235-80d7-4069-b37b-01ebeacda5d7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:43.417451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 574/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:43 2022] GET /identity/v3/role_assignments?user.id=fec6326157a043e09236360eb175d76c&scope.project.id=82cea0f40d654bb18fddf6b0854a3adc&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:45.450253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0a93-18ea-49cd-a158-dd7bbe99b41b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:45.450836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0a93-18ea-49cd-a158-dd7bbe99b41b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:45.450836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a0a93-18ea-49cd-a158-dd7bbe99b41b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:45.452271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 588/1162] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:45.458414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:45.458840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:45.458973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:45.475439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:45.476699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:45.543241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 575/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:41:45.566669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a0a93-18ea-49cd-a158-dd7bbe99b41b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:45.667672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a0a93-18ea-49cd-a158-dd7bbe99b41b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:45.703521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:45.704091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:45.704378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:45.704656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:45.771755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:45.774374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:45.776136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:45.872708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 589/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:45.878859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:45.973171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b8ec-f36e-443d-a2ac-258bad700199 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:45.986889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:46.008401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:46.009202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:46.009484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:46.009758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:46.016456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:46.017895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-4d2bfc24-21ef-4ee1-893a-c5895ce94215 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:46.032440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:46.033754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:41:46.034684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 576/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:41:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:46.040480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:46.133238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:46.136441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b5a3-b796-46ce-819a-698b750a955f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:46.170659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:46.171315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:46.171598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:46.171870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:46.172578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:46.172880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:46.174233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116303f-6226-497d-8fe2-9f8c01d05a46 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:46.182050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 590/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jun 30 17:41:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:48.203668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d989-e6c9-4fd6-bf28-6eb103c3af62 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:48.204195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d989-e6c9-4fd6-bf28-6eb103c3af62 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:48.204195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d989-e6c9-4fd6-bf28-6eb103c3af62 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:48.205595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 577/1167] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:48.211121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f704b-07c4-4e26-8169-81cf1aa46aef None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:48.211583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f704b-07c4-4e26-8169-81cf1aa46aef None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:48.211673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f704b-07c4-4e26-8169-81cf1aa46aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:48.228122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fe9f704b-07c4-4e26-8169-81cf1aa46aef None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:48.229327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fe9f704b-07c4-4e26-8169-81cf1aa46aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:48.295180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 591/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:48.318758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4d989-e6c9-4fd6-bf28-6eb103c3af62 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:48.418884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4d989-e6c9-4fd6-bf28-6eb103c3af62 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:48.452337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:48.452988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:48.453266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:48.453538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:48.454259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:48.455642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68370e18-ae0a-4278-8171-25820291a065 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:48.470679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 578/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Jun 30 17:41:48 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:50.536973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f15b1-10af-4079-b207-6488a3f7fa7b None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:50.538125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f15b1-10af-4079-b207-6488a3f7fa7b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:50.538125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f15b1-10af-4079-b207-6488a3f7fa7b None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:50.539505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 592/1170] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:50.544620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:50.545048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:50.545175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:50.561500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:50.562712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:50.587988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:50.627862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 579/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:50.651419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f15b1-10af-4079-b207-6488a3f7fa7b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:50.744483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:50.752426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f15b1-10af-4079-b207-6488a3f7fa7b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:50.785678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:50.786551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:50.786837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:50.787159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] PATH_INFO: `/v3/domains/9324b2a75b4248358bd713cd667d9370` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:50.797113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=9324b2a75b4248358bd713cd667d9370)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:50.798578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2015e-4cd3-4fc0-a024-d889e391c33c None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:50.800203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 593/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jun 30 17:41:50 2022] GET /identity/v3/domains/9324b2a75b4248358bd713cd667d9370 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:50.806317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:50.899958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171d9c0-862d-48da-a0cb-9d51bc9c1476 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:50.931210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:50.931849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:50.932128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:50.932398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:50.933081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:50.934633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20856922-8096-4cea-9375-5da5181951f7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:50.959995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 580/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:41:50 2022] POST /identity/v3/projects => generated 361 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:53.005819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbaaf8-a206-423e-9967-1b482af9c9bf None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:53.006887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbaaf8-a206-423e-9967-1b482af9c9bf None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:53.007035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbaaf8-a206-423e-9967-1b482af9c9bf None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:53.008410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 594/1174] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:53.013561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:53.013988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:53.014126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:53.030759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:53.031977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:53.042649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:53.101974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 581/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:53.127799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbaaf8-a206-423e-9967-1b482af9c9bf None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:53.230653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbaaf8-a206-423e-9967-1b482af9c9bf None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:53.264271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:53.265052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:53.265338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:53.265614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] PATH_INFO: `/v3/domains/9324b2a75b4248358bd713cd667d9370` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:53.266821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9324b2a75b4248358bd713cd667d9370)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:53.268261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d861ff-724d-417b-919d-fe5292e0f3b2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:53.269904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 595/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jun 30 17:41:53 2022] GET /identity/v3/domains/9324b2a75b4248358bd713cd667d9370 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:53.275833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:53.371461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334afcfe-8660-4eb0-8635-3844510acc2b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:53.406075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:53.406876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:53.407170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:53.407450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:53.408148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:53.409576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5da27-7f3c-4c58-bf0c-724fde4471c0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:53.428220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:53.433978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 582/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:41:53 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:41:55.513002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0032e1-daed-4ca2-98fa-854240cba381 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:55.513557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0032e1-daed-4ca2-98fa-854240cba381 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:55.513557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0032e1-daed-4ca2-98fa-854240cba381 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:55.514942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 596/1178] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:55.520148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:55.520539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:55.520697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:55.537684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:55.538915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:55.602723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 583/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:55.625698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0032e1-daed-4ca2-98fa-854240cba381 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:55.651117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:55.731481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0032e1-daed-4ca2-98fa-854240cba381 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:55.783987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:55.784700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:55.784985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:55.785261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:55.795717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:55.803763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:55.804931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:55.806061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 597/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:55.811074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:55.905631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854d29-9fae-469c-beca-def9eae2346a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:55.932878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:55.938806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:55.939448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:55.939730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:55.940004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:55.940737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:55.941041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:55.942424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:55.951065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 584/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:55.956760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:56.035668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:56.054320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296aa9-16b9-44a3-b827-411e288cecc5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:56.089182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:56.090054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:56.090354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:56.090652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] PATH_INFO: `/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:56.104610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] RBAC: Authorizing `identity:get_user(user_id=3f98ccc1796f4e4c9fa57cad4231dc7f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:56.106065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:56.107837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 598/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:55 2022] GET /identity/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:56.114209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:56.214172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814b059-6b77-403d-85d9-efaa729f8fd0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:56.251146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:56.251806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:56.252089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:56.252367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:56.310802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:56.320645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:56.322090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:56.423739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 585/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:41:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:41:56.429623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:56.525498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e580e56-9424-43f3-aeee-86cd0d73ea29 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:56.557813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:56.558676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:56.558954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:56.559221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] PATH_INFO: `/v3/projects/7ef2d08f319f4fd78974cfb978378b0a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:56.560368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] RBAC: Authorizing `identity:get_project(project_id=7ef2d08f319f4fd78974cfb978378b0a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:56.561827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732797b3-9472-4cf2-888c-1b154cc11962 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:56.563472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 599/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:56 2022] GET /identity/v3/projects/7ef2d08f319f4fd78974cfb978378b0a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:56.569781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:56.655129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:56.664865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0554dc-e3c6-4c36-b143-301db35527e4 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:56.697350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:56.698014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:56.698299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:56.698610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:56.699423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=7ef2d08f319f4fd78974cfb978378b0a, user.id=3f98ccc1796f4e4c9fa57cad4231dc7f) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:56.699734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:56.701155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72e64-8ba4-44fc-bd3c-496c9c0b2412 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:56.706449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 586/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:41:56 2022] GET /identity/v3/role_assignments?user.id=3f98ccc1796f4e4c9fa57cad4231dc7f&scope.project.id=7ef2d08f319f4fd78974cfb978378b0a&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:58.690283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53497faa-7572-4531-ae3d-e3896ba7e89e None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:58.690920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53497faa-7572-4531-ae3d-e3896ba7e89e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:58.691119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53497faa-7572-4531-ae3d-e3896ba7e89e None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:58.692592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 600/1186] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:41:58.698565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:58.699167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:58.699167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:58.716270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:41:58.717468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:41:58.784202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 587/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:41:58.807434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53497faa-7572-4531-ae3d-e3896ba7e89e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:58.818377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:58.907692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53497faa-7572-4531-ae3d-e3896ba7e89e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:58.940748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:58.941451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:58.941734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:58.942012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:58.952109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:41:58.960150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:41:58.961113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:41:58.962184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 601/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:41:58.967420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:59.048436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:59.062980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c78b13-8086-4b3f-8f94-6b8eeaabafc7 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:59.106526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:59.107404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:59.107817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:59.108198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:59.109239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:41:59.109722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:59.111646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:59.122677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 588/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:59.128400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:59.194714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:59.228334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d012c-411e-431c-b3c3-69c5cfe95ca3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:59.261182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:59.262043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:59.262349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:59.262634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] PATH_INFO: `/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:59.263819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] RBAC: Authorizing `identity:get_user(user_id=3f98ccc1796f4e4c9fa57cad4231dc7f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:59.265259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:59.266940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 602/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:41:59 2022] GET /identity/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:59.273003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:59.370498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea638-82bc-4b81-a6c7-863ed37dc01d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:59.402136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:59.403003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:59.403285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:59.403560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] PATH_INFO: `/v3/projects/7ef2d08f319f4fd78974cfb978378b0a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:59.404707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=7ef2d08f319f4fd78974cfb978378b0a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:59.406124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fad4e-73dc-4847-bd80-5ec3dad76bc5 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:59.407825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 589/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:41:59 2022] GET /identity/v3/projects/7ef2d08f319f4fd78974cfb978378b0a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:41:59.414556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:41:59.511058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027570b8-24b2-451a-bd53-338ff70343c1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:41:59.544664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:41:59.545532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:41:59.545811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:41:59.546082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] PATH_INFO: `/v3/projects/7ef2d08f319f4fd78974cfb978378b0a/users/3f98ccc1796f4e4c9fa57cad4231dc7f/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:41:59.548033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ef2d08f319f4fd78974cfb978378b0a, user_id=3f98ccc1796f4e4c9fa57cad4231dc7f, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:41:59.549437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa676a-c882-46de-b48c-b48828dfbf14 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:41:59.552402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:41:59.564583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 603/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:41:59 2022] PUT /identity/v3/projects/7ef2d08f319f4fd78974cfb978378b0a/users/3f98ccc1796f4e4c9fa57cad4231dc7f/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:42:01.558084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf53a-baaf-4cf6-b256-95245800fb99 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:01.558638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf53a-baaf-4cf6-b256-95245800fb99 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:01.558638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacf53a-baaf-4cf6-b256-95245800fb99 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:01.560031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 590/1193] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:42:01.565273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:01.565674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:01.565839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:01.583296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:42:01.585793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:42:01.651557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 604/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:42:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:42:01.675146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf53a-baaf-4cf6-b256-95245800fb99 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:01.694724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:01.777717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacf53a-baaf-4cf6-b256-95245800fb99 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:01.810645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:01.811346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:01.811624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:01.811893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:01.822543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:42:01.830072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:42:01.831206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:42:01.832198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 591/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:42:01.837703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:01.932330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db29674-e0da-41bf-a8f8-bcb29a829219 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:01.959305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:01.968380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:01.969038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:01.969316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:01.969585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:01.970317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:42:01.970648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:01.971999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:01.980051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 605/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:42:01.985868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:02.061142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:02.083659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4be84c-36fa-42ea-b4e0-62786d8519ac None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:02.116264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:02.117111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:02.117397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:02.117676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] PATH_INFO: `/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:02.118866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] RBAC: Authorizing `identity:get_user(user_id=3f98ccc1796f4e4c9fa57cad4231dc7f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:02.120299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:02.121920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 592/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jun 30 17:42:01 2022] GET /identity/v3/users/3f98ccc1796f4e4c9fa57cad4231dc7f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:42:02.127652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:02.222679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37cf35-4c98-4487-aded-49c0c165d544 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:02.255187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:02.255846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:02.256131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:02.256405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:02.308745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:02.321469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:02.322874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:02.416452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 606/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:42:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:42:02.422260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:02.518021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7be78-9e80-480c-82b5-291f9826e64e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:02.554862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:02.555714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:02.555995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:02.556266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] PATH_INFO: `/v3/projects/7ef2d08f319f4fd78974cfb978378b0a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:02.557437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] RBAC: Authorizing `identity:get_project(project_id=7ef2d08f319f4fd78974cfb978378b0a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:02.558897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35441aa2-1bd8-48c9-9ee9-bbf5bab183c7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:02.560593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 593/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jun 30 17:42:02 2022] GET /identity/v3/projects/7ef2d08f319f4fd78974cfb978378b0a => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:42:02.566706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:02.649860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:02.661596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490c4ab-311c-4969-b6ec-058027e64e46 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:02.694145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:02.694798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:02.695076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:02.695344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:02.696146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=7ef2d08f319f4fd78974cfb978378b0a, user.id=3f98ccc1796f4e4c9fa57cad4231dc7f) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:42:02.696452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:02.697849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ead0c-b958-42a6-80f8-4f3bb15b51a3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:02.703451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 607/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:42:02 2022] GET /identity/v3/role_assignments?user.id=3f98ccc1796f4e4c9fa57cad4231dc7f&scope.project.id=7ef2d08f319f4fd78974cfb978378b0a&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:42:12.550727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:12.700899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=24 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:58.342378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51a40-c048-445d-8a7b-118a08a1a065 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:58.342378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51a40-c048-445d-8a7b-118a08a1a065 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:58.342378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce51a40-c048-445d-8a7b-118a08a1a065 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:58.343376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 594/1201] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:42:58.349776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:58.350297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:58.350297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:58.373026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:42:58.374373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:42:58.468260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 608/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:42:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:42:58.494050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce51a40-c048-445d-8a7b-118a08a1a065 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:58.623629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce51a40-c048-445d-8a7b-118a08a1a065 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:58.664438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:58.666392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:58.666392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:58.666392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:58.687623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:58.689181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:42:58.712446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:42:58.713945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:42:58.721121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:42:58.721121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:42:58.721121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:42:58.721121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:42:58.721121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 595/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:42:58.721622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:58.866988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b816a86-dacd-4fc8-8db2-c0f978403740 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:58.901722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:58.909381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:58.910181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:58.910582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:58.910874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:58.911601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:42:58.911969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:58.913585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44322e-2506-44a3-ab2b-97f977e39269 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:58.927894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 609/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:42:58.934779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:42:59.010709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:42:59.084829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2eb49-8329-4daf-aa1a-be55bb4ecdce None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:42:59.129702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:42:59.130538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:42:59.130833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:42:59.131114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:42:59.131931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:42:59.133504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523cf8e-77b1-42e1-83f9-5313d49cc9b4 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:42:59.166454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 596/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jun 30 17:42:58 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:43:01.274771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cbfd9-3f34-4b5b-b78e-ff88569c922c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:01.275274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cbfd9-3f34-4b5b-b78e-ff88569c922c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:01.275274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cbfd9-3f34-4b5b-b78e-ff88569c922c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:01.276920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 610/1206] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:01.284360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:01.284810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:01.285239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:01.305014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:01.306325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:01.381077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 597/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:01.410003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cbfd9-3f34-4b5b-b78e-ff88569c922c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:01.534835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cbfd9-3f34-4b5b-b78e-ff88569c922c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:01.572128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:01.572926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:01.573207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:01.573484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:01.584485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:01.592846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:01.593857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:01.594758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 611/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:01.601448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:01.647635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:01.724665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af069ed-6608-4513-9c60-b437ceeff026 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:01.764603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:01.765338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:01.765724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:01.766077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:01.767424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:01.767753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:01.769752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:01.782363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 598/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:01.785382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:01.841885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:01.899396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5f3e4-1889-4778-b7b9-43671c70a075 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:01.939689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:01.940497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:01.940773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:01.941039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:01.952341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:01.953951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:01.973450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:01.974685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:01.980458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:01.980458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:01.980458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:01.980458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:01.980458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 612/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:01.980824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:02.093610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:02.097455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ad1bc-ec71-43c9-9a7b-15c0ee638923 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:02.137913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:02.138558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:02.138844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:02.139122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:02.139868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:02.140184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:02.141561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:02.158065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 599/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:02.163756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:02.280029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd432c-0660-4429-b54a-985d4a654365 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:02.281641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:02.318967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:02.319941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:02.320072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:02.320321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:02.329209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:02.331201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:02.346277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:02.347486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:02.353119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 613/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:02.353751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:02.502527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e250424-e824-4590-bf4e-e990e68a1061 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:02.542243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:02.542972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:02.543283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:02.543775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:02.544689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:02.544999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:02.546435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:02.567766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 600/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:02.574008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:02.691956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0848e55-e86f-4aa1-b8e8-e3d2f0987ad3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:02.741660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:02.742634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:02.743015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:02.744012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:02.778363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:02.822547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:02.824327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:02.938889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 614/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:43:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 17:43:02.946058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:02.968538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:03.076303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb06b25-65ca-42fc-84a4-54c2170f4e70 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:03.118836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:03.119784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:03.120160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:03.120535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:03.133940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:03.135452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:03.153731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:03.155254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:03.156189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:03.156189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:03.156189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:03.156189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:03.156189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 601/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:03.161685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:03.229544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:03.269857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085f05-f201-47b9-8488-0dd45a977834 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:03.307036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:03.307710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:03.307971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:03.308258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:03.308953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:03.309259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:03.310672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:03.325085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 615/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:03.331579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:03.403138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:03.442499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcad18-8a8b-4251-9973-bb3b174a8279 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:03.478620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:03.479416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:03.479749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:03.479978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:03.487048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:03.488508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:03.504258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:03.506106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:03.507434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 602/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:03.512231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:03.628823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeec43-5e0f-497a-9160-e5aaebcc0c60 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:03.653181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:03.665626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:03.666476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:03.666862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:03.667238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:03.668120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:03.668530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:03.670045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d41cb-1e8c-4ed0-996c-5e9d3a6e1805 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:03.684015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 616/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:03.690705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:03.802525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d55cf-222b-4527-ab18-095711488e95 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:03.834477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:03.841311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:03.842060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:03.842412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:03.842691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:03.843701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=1f20dd9dac7a4c4bb649e484d53a4928) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:03.844035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:03.845596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4ccd9-b1ee-49cb-b67f-49ef297db2d2 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:03.851923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 603/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:43:03 2022] GET /identity/v3/role_assignments?user.id=1f20dd9dac7a4c4bb649e484d53a4928&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:05.871283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d92b-cec0-4ed4-b833-106bab36e630 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:05.872035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d92b-cec0-4ed4-b833-106bab36e630 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:05.872035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d92b-cec0-4ed4-b833-106bab36e630 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:05.873190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 617/1220] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:43:05.879356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:05.879865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:05.880183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:05.897214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:05.898472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:05.972037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 604/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:05.994732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115d92b-cec0-4ed4-b833-106bab36e630 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.110238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115d92b-cec0-4ed4-b833-106bab36e630 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:06.149227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:06.150002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:06.150321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:06.150633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:06.164014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:06.172494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:06.174212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:06.180338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 618/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:06.181198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.287364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0034779-92c2-4508-b743-35a9a31e734c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:06.313029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:06.326816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:06.327571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:06.327889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:06.328268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:06.329292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] RBAC: Adding query filter params (name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:06.329688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:06.331300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:06.339785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 605/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:06.345735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.391687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:06.454520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919b5ff-8b33-41dd-b75e-33449d11ae4d None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:06.488283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:06.489069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:06.489351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:06.489622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:06.500446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:06.502551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:06.521397 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:06.522603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:06.523725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:06.523725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:06.523725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:06.523725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:06.523725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 619/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:06.528990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.637806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b94a2b-ec5c-4630-bc1b-a185ee58c617 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:06.676444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:06.677100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:06.677386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:06.677663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:06.678412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:06.678724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:06.680081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:06.691984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 606/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:06.697979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.803986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:06.812221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543f585-e768-44ba-ac5e-eaa64c4bc218 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:06.846561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:06.847371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:06.847656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:06.847932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:06.857880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:06.859775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:06.875557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:06.877195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:06.878316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 620/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:06.882928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:06.931402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:06.988446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b20e23-a082-499a-b173-d6ff8fea3b54 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.023966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.024726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.025008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.025281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.026107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:07.026442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.027861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.045628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 607/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:07.051477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:07.162266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ed15f-d585-4fe1-b5d9-f9af2e03c8d5 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.197445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.198417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.198527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.198803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.211375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.212814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:07.230324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:07.231398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:07.232700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:07.232700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:07.232700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:07.232700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:07.232700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 621/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:07.237338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:07.342662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8e2d-6f53-44fb-9911-6517b173e48d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.344965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:07.380677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.381374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.381622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.381878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.382643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:07.382926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.384315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.396668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 608/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:07.402506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:07.411897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:07.508334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e679-c238-44d5-8525-852550ca3cd3 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.542183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.543010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.543291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.543565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.550302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.551834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:07.570393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:07.571555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:07.572692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 622/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:43:07.580098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:07.698385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346a101-2170-4a0c-b514-be93fd1f93e6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.737554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.738241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.738552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.738827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.739638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:07.739928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.741395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebb63-36f1-487e-8396-a4f6e0fc8095 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.754875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 609/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:07.760827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:07.832129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:07.870041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3173f-1012-415a-9365-edd678bb94c8 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:07.905085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64406171-0602-4724-994f-784af95a8c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:07.905938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406171-0602-4724-994f-784af95a8c90 None admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:07.906226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406171-0602-4724-994f-784af95a8c90 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:07.906527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64406171-0602-4724-994f-784af95a8c90 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/1f20dd9dac7a4c4bb649e484d53a4928/roles/192ae45574f74568b13dcfdde2d5d2a2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:07.920945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64406171-0602-4724-994f-784af95a8c90 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=1f20dd9dac7a4c4bb649e484d53a4928, role_id=192ae45574f74568b13dcfdde2d5d2a2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:07.922405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64406171-0602-4724-994f-784af95a8c90 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:07.935782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 623/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:43:07 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/1f20dd9dac7a4c4bb649e484d53a4928/roles/192ae45574f74568b13dcfdde2d5d2a2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:43:09.958934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d1c4-299d-4432-b6a0-beb76a85637b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:09.959479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d1c4-299d-4432-b6a0-beb76a85637b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:09.959479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d1c4-299d-4432-b6a0-beb76a85637b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:09.960851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 610/1233] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:09.966053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:09.966500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:09.966628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:09.983802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:09.985024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:10.051859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 624/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:10.075644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5d1c4-299d-4432-b6a0-beb76a85637b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:10.102679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:10.178767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5d1c4-299d-4432-b6a0-beb76a85637b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:10.218410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:10.219836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:10.220193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:10.220706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:10.231357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:10.239021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:10.240203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:10.246018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 611/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:10.247289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:10.296964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:10.358812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca24b0c-a654-4c78-ada5-549eb431d49f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:10.396442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:10.397107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:10.397389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:10.397666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:10.398437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:10.398745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:10.400115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:10.408667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 625/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:10.414591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:10.506901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:10.523744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0213db-d89f-4d5b-b5d7-a8d86330b01d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:10.557159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:10.557942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:10.558220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:10.558518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:10.571938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:10.573728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:10.594135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:10.595285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:10.600909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:10.600909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:10.600909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:10.600909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:10.600909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 612/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:10.602341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:10.707675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df2ef2-166d-448e-a2a7-908d9639ffa1 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:10.717641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:10.746269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:10.746996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:10.747308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:10.747598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:10.748406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:10.748737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:10.750248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:10.769248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 626/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:10.770404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:10.879147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7847-84c0-46d2-be6c-0546bb69a82d None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:10.916257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:10.917055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:10.917396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:10.917673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:10.920594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:10.926976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:10.928447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:10.943451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:10.944426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:10.945427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 613/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:43:10.950514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:11.057599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2c157-54d1-419f-ad8c-45d384d5713a None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:11.093277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:11.093939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:11.094777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:11.095129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:11.095979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:11.096211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:11.097722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:11.116277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 627/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:11.122470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:11.227671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0387-0338-4ab7-a68f-9fcd305a4f59 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:11.263680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:11.264565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:11.264998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:11.265487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:11.332829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:11.340167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:11.341817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:11.447473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 614/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:43:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:43:11.453240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:11.482376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:11.559747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0ec7-cd9b-4dda-b1d6-6213c9cbd595 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:11.595125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:11.595819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:11.596097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:11.596364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:11.606053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:11.607522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:11.626665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:11.627710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:11.628615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:11.628615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:11.628615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:11.628615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:11.628615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 628/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:11.633993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:11.738558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:11.745195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f6e68-49e3-4f06-a574-70c211ba7961 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:11.788175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:11.788872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:11.789221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:11.789497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:11.790587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:11.790897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:11.792492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:11.805742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 615/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:11.811656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:11.899395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:11.923806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f09de-04d4-4549-beb6-c13df70e5879 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:11.962617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:11.963568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:11.963849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:11.964177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:11.971898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:11.973542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:11.988442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:11.989465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:11.990478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 629/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:43:11.995696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:12.101834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634b621-543a-4618-b362-70acc78e1a40 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:12.135252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:12.135962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:12.136171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:12.136442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:12.137207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:12.137513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:12.138963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc0841-a6e0-4c44-bd39-4a6e4dabb6f8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:12.153539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 616/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:12.159684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:12.272264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69fe45-c521-481d-ba80-ad2ecec17fdf None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:12.306207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:12.306889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:12.307181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:12.307448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:12.308283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] RBAC: Adding query filter params (role.id=192ae45574f74568b13dcfdde2d5d2a2, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=1f20dd9dac7a4c4bb649e484d53a4928) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:12.308571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:12.310026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4bd96-f13a-490b-8d65-f7ebd1ada6bc None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:12.317011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 630/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:43:12 2022] GET /identity/v3/role_assignments?user.id=1f20dd9dac7a4c4bb649e484d53a4928&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=192ae45574f74568b13dcfdde2d5d2a2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:14.573614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1a04-1983-496a-b7c4-c55054760554 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:14.574190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1a04-1983-496a-b7c4-c55054760554 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:14.574190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf1a04-1983-496a-b7c4-c55054760554 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:14.575602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 617/1247] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:14.581825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:14.582247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:14.582480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:14.587770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:14.601213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:14.602475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:14.676366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 631/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 17:43:14.702581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf1a04-1983-496a-b7c4-c55054760554 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:14.726196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:14.832187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf1a04-1983-496a-b7c4-c55054760554 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:14.875019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:14.876586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:14.876903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:14.877236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:14.888777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:14.897622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:14.898664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:14.899791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 618/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:14.905538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.011229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6bad0-da89-4ea0-801e-edd22a63ee28 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.048285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:15.054016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.054684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.055007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.055323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:15.056138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:15.056453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:15.057837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:15.067332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 632/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:15.073142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.156332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:15.180175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1a0aa-fd61-4fb7-afa2-cb28353e503b None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.219113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.220096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.220520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.220905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:15.232484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:15.234013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:15.253792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:15.254975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:15.256262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:15.256262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:15.256262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:15.256262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:15.256262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 619/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:15.261090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.372492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a479e9-d395-4da5-be3c-9343c0dd0765 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.408735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.409328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.409616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.409892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:15.410653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:15.410962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:15.412335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:15.425434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 633/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:15.431588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.542262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba2c3b-053f-4eb0-aee9-66ae51f31ce0 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.571126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:15.579952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.580806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.581166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.581442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:15.589479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:15.590942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:15.605754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:15.606923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:15.607980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 620/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:15.613531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.675008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:15.727314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362bbaf-d60b-42fc-87b9-032bda9f62b0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.768064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.768786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.769101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.769395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:15.770314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:15.770705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:15.772269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:15.792080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 634/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:15.798587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:15.909902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee500f1-af19-4d46-adce-4d46321910d8 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:15.943790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:15.944337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:15.944621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:15.944894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.002160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:16.019933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.021377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.133812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 621/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:43:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:43:16.139896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:16.263734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7281b8a-ffc2-4f0e-8caa-9b9adb666ae2 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:16.277622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:16.301932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:16.302768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:16.303050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:16.303331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.313817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.315372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:16.333864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:16.334986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:16.336292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:16.336292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:16.336292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:16.336292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:16.336292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 635/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:16.340921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:16.428009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:16.439931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82159-283b-4c3e-8ea4-ae97f41a7703 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:16.475298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:16.475929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:16.476209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:16.476484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.477183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:16.477486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.478879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.490644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 622/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:16.496535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:16.605934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235d448-0795-4d23-997d-8d715447d0e9 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:16.641621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:16.642443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:16.642725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:16.643001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.649838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.651349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:16.665985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:16.667153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:16.668624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 636/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 17:43:16.673144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:16.780132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae5a0a-a7cf-4b24-837d-ffa901218485 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:16.816204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:16.816885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:16.817200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:16.817480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.818244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:16.818578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.819979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e9746-2b83-41fe-a0f5-824895a08bd8 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.832407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 623/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:16.838766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:16.866967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:16.950168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94974ee-5456-498a-859b-a727de7296ea None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:16.988233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:16.988922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:16.989176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:16.989432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:16.990521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=1f20dd9dac7a4c4bb649e484d53a4928) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:16.990733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:16.992222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b253c50-7161-4937-a798-6cfdeb752a1d None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:16.998243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 637/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:43:16 2022] GET /identity/v3/role_assignments?user.id=1f20dd9dac7a4c4bb649e484d53a4928&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:18.996194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fbcc-4960-4e22-9787-c3966fa2e179 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:18.996642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fbcc-4960-4e22-9787-c3966fa2e179 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:18.996756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fbcc-4960-4e22-9787-c3966fa2e179 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:18.998145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 624/1261] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:19.003262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.003655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.003813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.020855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:19.022083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:19.086693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 638/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:19.110022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fbcc-4960-4e22-9787-c3966fa2e179 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.120709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:19.207666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fbcc-4960-4e22-9787-c3966fa2e179 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:19.239875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:19.240698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.240933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.241207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.251401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:19.258890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:19.259994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:19.261053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 625/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:19.266890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.284419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:19.362324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d3208-7bbc-4c6f-87ce-2b0432a6f3fc None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:19.396385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:19.397037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.397323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.397600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.398350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:19.398658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:19.400021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:19.407882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 639/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:19.413508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.477338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:19.509560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85752e56-9974-4c25-9c7d-e71a8e97ab99 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:19.541425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:19.542192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.542504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.542780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.552821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:19.554252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:19.571702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:19.572758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:19.573968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:19.573968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:19.573968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:19.573968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:19.573968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 626/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:19.579207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.657940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:19.677777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1a0a-20a9-4ce6-bfed-92357d7eba13 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:19.712810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:19.713525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.713809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.714116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.714867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:19.715170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:19.716598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:19.728957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 640/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:19.735632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.837330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75538c-9614-4a61-8181-4e7a65fc49fc None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:19.855733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:19.869689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:19.870502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:19.870785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:19.871056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] PATH_INFO: `/v3/users/placement` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:19.878845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:19.880278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:19.894068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:19.895195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:19.896364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 627/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:19.901006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:19.998606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b24b5-2db9-46b4-9994-bbd735d3ab3f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.031898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.032550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.032827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.033095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.033907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:20.034215 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.035647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.043923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:20.052830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 641/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:20.058427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:20.153708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3ed31-da91-4aae-a597-9e529bf6442e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.186092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.186890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.187231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.187522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.197858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.199329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:20.216922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:20.217960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:20.219583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:20.219583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:20.219583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:20.219583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:20.219583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 628/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:20.224147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:20.321175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107744-87ce-4a24-9b40-b30443da5786 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.353879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.354678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.355043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.355363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.356069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:20.356368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.357719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.369061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 642/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:20.374687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:20.405430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:20.472141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81d07-96b4-4695-bad0-279a5ac6b0ba None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.504410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.505175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.505452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.505720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.512095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.513511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:20.527911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:20.529058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:20.530441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 629/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:20.535458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:20.591521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:20.631630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fe4cf-40c9-4576-816d-080a2d54e37f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.664325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.664908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.665187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.665457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.666218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:20.666548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.667960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a20101-c907-44a0-8da9-828b1daf6b73 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.679568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 643/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:20.685379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:20.781993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:20.790607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0ec45-8ffa-4bce-9775-2ed8949caa6e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:20.823358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:20.824227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:20.824509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:20.824784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] PATH_INFO: `/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/1f20dd9dac7a4c4bb649e484d53a4928/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:20.826717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4089a869f84cb59bf14cbf816a8fd3, user_id=1f20dd9dac7a4c4bb649e484d53a4928, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:20.828796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88287-7663-4a43-a483-ee56efa3a978 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:20.840784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 630/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jun 30 17:43:20 2022] PUT /identity/v3/projects/8a4089a869f84cb59bf14cbf816a8fd3/users/1f20dd9dac7a4c4bb649e484d53a4928/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:43:22.846270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ce66-9b4e-4597-975b-90ee5a465515 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:22.846849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ce66-9b4e-4597-975b-90ee5a465515 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:22.846849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ce66-9b4e-4597-975b-90ee5a465515 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:22.848228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 644/1274] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:22.853372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:22.853779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:22.853932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:22.870643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:22.871823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:22.934456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 631/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:22.960159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ce66-9b4e-4597-975b-90ee5a465515 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.059870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ce66-9b4e-4597-975b-90ee5a465515 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.076460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:23.092953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.094257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.094564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.094838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.104633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 17:43:23.113816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:23.115042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:23.120806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 645/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jun 30 17:43:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:23.121835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.199022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:23.216457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bcbdc-f761-4bc6-83a7-25453941d2f9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.249223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.249902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.250184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.250505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.251225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:23.251531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:23.252889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:23.260701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 632/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:23.266344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.363024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f046-64a8-44c6-b755-0917902d455e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.394697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.395458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.395735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.396006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.406775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:23.408670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:23.424962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:23.426045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:23.427050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:23.427050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:23.427050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:23.427050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:23.427050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 646/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:23.431700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.526132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111f902-184a-4b4f-aba0-45e66da187e6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.558007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.558671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.558952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.559221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.559921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:23.560223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:23.561567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:23.572927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 633/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:23.579140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.609187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:23.675135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123be8f-1984-4936-ad9a-a76abd4d77fa None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.707321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.708095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.708375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.708644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.717392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:23.719498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 17:43:23.736431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:23.737606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:23.738841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 647/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:23.743631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.747414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:23.840786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176ce9e-9433-43ff-93ae-592486d51bef None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:23.873557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:23.874208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:23.875042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:23.875319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:23.876140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:23.876448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:23.877867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:23.894763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 634/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:23.900402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:23.988815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:23.999025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a64738-66b4-40f5-b53f-97c8d8459638 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.032428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.033076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.033362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.033642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.098170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.099630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.184055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:24.195331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 648/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jun 30 17:43:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 17:43:24.201032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:24.260644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:24.297511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1222e6-1de6-405c-bf2b-6a64f050d14e None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.330049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.330847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.331134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.331411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.340965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.342426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 17:43:24.359776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:24.360857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:24.366476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 17:43:24.366476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 17:43:24.366476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 17:43:24.366476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:24.366476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 635/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:24.366830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:24.460196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48ad2-8290-4956-bb51-54b17d097ef0 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.492517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.493157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.493436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.493707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.494430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:24.494735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.496078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.507455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 649/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jun 30 17:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:24.512942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:24.607474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11038d-f228-4b02-ba7d-29338b3218f6 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.621058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:24.640176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.640942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.641223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.641494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.647858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.649283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 17:43:24.663047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:24.664128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: ERROR keystone.server.flask.application Jun 30 17:43:24.665417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 636/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jun 30 17:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:24.669799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:24.720174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:24.773624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8785e-b64d-4b6d-94f7-01e360154dbf None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.806352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.807015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.807300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.807575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.808350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:24.808659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.810069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3199fe-26ba-45a1-a69a-f2ea4b6c4019 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.821942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 650/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jun 30 17:43:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:43:24.827919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:24.923943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3c3ea-c1b4-4b19-893f-65244e865039 None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:24.959979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:24.959979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:24.960268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:24.960458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:24.961579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] RBAC: Adding query filter params (role.id=1df4096f023c48ccab33e5939498b3da, scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3, user.id=1f20dd9dac7a4c4bb649e484d53a4928) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:24.961705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:24.963100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec7452-afa3-415c-873f-d463383737ec None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:24.971553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 637/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jun 30 17:43:24 2022] GET /identity/v3/role_assignments?user.id=1f20dd9dac7a4c4bb649e484d53a4928&scope.project.id=8a4089a869f84cb59bf14cbf816a8fd3&role.id=1df4096f023c48ccab33e5939498b3da => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:27.045192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1e6b-04da-4706-86f5-596d419a2fc6 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:27.045684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1e6b-04da-4706-86f5-596d419a2fc6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:27.045771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1e6b-04da-4706-86f5-596d419a2fc6 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:27.047244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 651/1288] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:27.052189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:27.052645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:27.052751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:27.069703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:27.070913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:27.099000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:27.135010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 638/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:27.157924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1e6b-04da-4706-86f5-596d419a2fc6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:27.261204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1e6b-04da-4706-86f5-596d419a2fc6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:27.272288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:27.295176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:27.295960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:27.296766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:27.297040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] PATH_INFO: `/v3/services/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:27.297676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:27.299066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:27.310501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:27.312293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:27.313373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 652/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:43:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:27.318413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:27.412917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c764c-a7be-4603-b071-cfd07a36211e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:27.447038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:27.447685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:27.447969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:27.448245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:27.448942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:27.449248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:27.450636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a0806-645b-4187-bc90-67b0b29d86a7 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:27.455347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 639/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 17:43:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:27.461034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:27.557490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc723c33-8802-4038-a440-334d1f61830e None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:27.590485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:27.591133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:27.591415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:27.591693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] PATH_INFO: `/v3/services` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:27.592398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] RBAC: Adding query filter params (type=placement) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:27.592708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:27.594597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d7c8-0b6a-4f4d-96cf-e3f5258179ba None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:27.599066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 653/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 17:43:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:29.591642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973372e-8cbf-442d-8197-75a25d3b7f97 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:29.592194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973372e-8cbf-442d-8197-75a25d3b7f97 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:29.592194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973372e-8cbf-442d-8197-75a25d3b7f97 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:29.594550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 640/1293] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:29.599764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd6b1-9d2a-4aae-85b3-3f0edd926eab None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:29.600149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd6b1-9d2a-4aae-85b3-3f0edd926eab None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:29.600326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dd6b1-9d2a-4aae-85b3-3f0edd926eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:29.619187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c09dd6b1-9d2a-4aae-85b3-3f0edd926eab None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:29.620402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c09dd6b1-9d2a-4aae-85b3-3f0edd926eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:29.651806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:29.689472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 654/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:29.712511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973372e-8cbf-442d-8197-75a25d3b7f97 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:29.750614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:29.824715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973372e-8cbf-442d-8197-75a25d3b7f97 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:29.857004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:29.857597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:29.857881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:29.858155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:29.859534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:29.860888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc363f-a01b-45b9-a2d8-e3f22c6b1b55 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:29.873278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 641/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jun 30 17:43:29 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:43:31.896482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213efb-af22-43fe-a05a-711d2eb70cca None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:31.897012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213efb-af22-43fe-a05a-711d2eb70cca None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:31.897012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213efb-af22-43fe-a05a-711d2eb70cca None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:31.898417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 655/1296] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:31.903416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:31.903798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:31.903968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:31.920721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:31.921891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:31.993921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 642/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:32.017787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213efb-af22-43fe-a05a-711d2eb70cca None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:32.115360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213efb-af22-43fe-a05a-711d2eb70cca None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:32.148736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:32.149445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:32.149727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:32.150000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] PATH_INFO: `/v3/services/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:32.150668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:32.152028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:32.163516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:32.164507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:32.165399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 656/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:43:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:32.170808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:32.188896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:32.268199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdd413-3929-4c0a-97db-e3d4ddd2215e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:32.301556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:32.302201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:32.302521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:32.302797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:32.303498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] RBAC: Adding query filter params (name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:32.303804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:32.305166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:32.310162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 643/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 17:43:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:32.316374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:32.347602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:32.414286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5fcb9-91dc-462e-86a7-c6eee3e9612f None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:32.446260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:32.446918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:32.447204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:32.447678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:32.448257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] RBAC: Adding query filter params (interface=public, service_id=9e10ee0d765842e6833b1b43ee5fcf47, region_id=RegionOne) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:32.448569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:32.449917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3371f37-a2a9-43ce-98b8-da989a3a2e78 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:32.455552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 657/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jun 30 17:43:32 2022] GET /identity/v3/endpoints?service_id=9e10ee0d765842e6833b1b43ee5fcf47&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:32.460998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:32.538671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:32.556371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380059-8fa0-491b-9aea-9e2a0dc2397a None admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:32.588679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:32.589317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:32.589598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:32.589871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:32.590562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:32.591911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d482e-7145-4389-a9e6-eee1c8296c51 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:32.599100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 644/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jun 30 17:43:32 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:43:34.628922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce6f7-13ec-4278-88ed-597772b97908 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:34.629452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce6f7-13ec-4278-88ed-597772b97908 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:34.629452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ce6f7-13ec-4278-88ed-597772b97908 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:34.630885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 658/1302] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:43:34.635926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:34.636335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:34.636475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:34.653118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:43:34.654281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:43:34.718464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 645/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:43:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 17:43:34.740770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ce6f7-13ec-4278-88ed-597772b97908 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:34.833254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:34.840251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ce6f7-13ec-4278-88ed-597772b97908 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:34.872880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:34.873666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:34.873951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:34.874227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:34.874898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:34.876250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 17:43:34.887876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 17:43:34.889052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application with Lock( Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self._enter() Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application return self.creator() Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application created_value = creator( Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: ERROR keystone.server.flask.application Jun 30 17:43:34.890680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 659/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jun 30 17:43:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 17:43:34.895498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:34.968262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:34.990628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54441477-c6fa-45d4-8653-ea316615bb24 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:35.024362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:35.025010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:35.025290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:35.025563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] PATH_INFO: `/v3/services` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:35.026254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:43:35.026589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:35.027945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836cd8-e4fd-4929-b5a8-c5c36f5b7a85 None admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:35.033122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 646/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jun 30 17:43:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:43:35.038821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:43:35.135434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f4b71-4df9-4538-8b44-290ecc481f54 None admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:43:35.167678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:43:35.168260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:43:35.168543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:43:35.168816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:43:35.169546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:43:35.170949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc37c0-798e-4b71-9d23-06a67b54645b None admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:43:35.188652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 660/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jun 30 17:43:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:43:45.030875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=30 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:43:45.185152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:37.766108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119912f5-03e1-4ecd-ba87-28026e3fe7b5 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:37.767175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119912f5-03e1-4ecd-ba87-28026e3fe7b5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:37.767699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119912f5-03e1-4ecd-ba87-28026e3fe7b5 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:37.773426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 647/1307] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:37.775061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5663-6990-4820-be11-e03f98927fb1 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:37.775501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5663-6990-4820-be11-e03f98927fb1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:37.775721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c5663-6990-4820-be11-e03f98927fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:37.876271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-dc7c5663-6990-4820-be11-e03f98927fb1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:37.879605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dc7c5663-6990-4820-be11-e03f98927fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:38.008525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 661/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:38.060154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:38.060451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:38.060732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:38.062527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 648/1309] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jun 30 17:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:38.074943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061a426-335f-45f5-b139-f2604ed732ea None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:38.075358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061a426-335f-45f5-b139-f2604ed732ea None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:38.075804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061a426-335f-45f5-b139-f2604ed732ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:38.167424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e061a426-335f-45f5-b139-f2604ed732ea None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:38.168638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e061a426-335f-45f5-b139-f2604ed732ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:38.195367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:38.271496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 662/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jun 30 17:44:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:44:38.280819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:38.282666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:38.399939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf403-9dfe-4233-9f2f-ce2027078953 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:38.480444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:38.481099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:38.481384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:38.481757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:38.483411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:38.542247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:38.599425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:38.600864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12698023-c978-4662-aac7-951ce6a208ae service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:38.621981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 649/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:40.941331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:40.941909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:40.941909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:40.943297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 663/1312] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:40.948978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686191-4ea2-4637-80c7-d3ccd6d0cb31 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:40.949369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686191-4ea2-4637-80c7-d3ccd6d0cb31 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:40.949538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686191-4ea2-4637-80c7-d3ccd6d0cb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:41.019699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-39686191-4ea2-4637-80c7-d3ccd6d0cb31 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:41.020925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-39686191-4ea2-4637-80c7-d3ccd6d0cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:41.120687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 650/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:41.159211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:41.160777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:41.273549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9caf70-3a68-4c10-899c-76642c566f0a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:41.347533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:41.349855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:41.350572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:41.351032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:41.351476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:41.353301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:41.461999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:41.463485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c9748-12f2-411d-9ab3-c71769b20465 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:41.481928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 664/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:43.906745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:43.907301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:43.907301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:43.908830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 651/1315] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:43.913728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77cc13-b034-46d4-a823-e91127d9102e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:43.914150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77cc13-b034-46d4-a823-e91127d9102e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:43.914286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77cc13-b034-46d4-a823-e91127d9102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:43.979229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ef77cc13-b034-46d4-a823-e91127d9102e None None] MFA Rules not processed for user `5d213696814d4a9aaf5ee9a25ff03fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:43.980448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ef77cc13-b034-46d4-a823-e91127d9102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:44.032986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:44.080830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 665/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:44.120389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:44.122026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:44.132343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:44.226925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77551b2-3473-4a5d-8b26-ece94d650626 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:44.299467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:44.300164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:44.300447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:44.300723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:44.302458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:44.367644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:44.414153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:44.415612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81ebe-04f5-4e6f-8095-a95922c38b36 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:44.434990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 652/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:47.034250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:47.034813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:47.034911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:47.036201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 666/1318] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:47.041524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d43e-a012-48d1-8a3f-9d85ed2f4612 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:47.041953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d43e-a012-48d1-8a3f-9d85ed2f4612 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:47.042088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d43e-a012-48d1-8a3f-9d85ed2f4612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:47.105049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4076d43e-a012-48d1-8a3f-9d85ed2f4612 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:47.106245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4076d43e-a012-48d1-8a3f-9d85ed2f4612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:47.199381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 653/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:47.236971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:47.238529 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:47.341445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97de9fb-8027-47ce-8341-20336f6c5978 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:47.411300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:47.411938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:47.412221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:47.412496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:47.414163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:47.422000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:47.521018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:47.522476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb9499-7132-4f3a-9746-afee1a789997 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:47.539895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 667/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:50.219081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:50.219628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:50.219628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:50.221026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 654/1321] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:50.226356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66e34c-39b9-48bd-82e9-e703a1417149 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:50.226879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66e34c-39b9-48bd-82e9-e703a1417149 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:50.226992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66e34c-39b9-48bd-82e9-e703a1417149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:50.291790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4d66e34c-39b9-48bd-82e9-e703a1417149 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:50.292929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4d66e34c-39b9-48bd-82e9-e703a1417149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:50.354024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:50.399039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 668/1322] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:50.437617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:50.439200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:50.461218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:50.550442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb50ef-d589-4eee-92cd-41a206927185 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:50.632209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:50.632866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:50.633149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:50.633420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:50.635151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:50.708171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:50.746963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:50.748325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6614-93a3-451b-b5e1-4c660bf775eb service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:50.767003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 655/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:52.871939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:52.872626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:52.872626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:52.873747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 669/1324] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:52.878918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401291-cb68-4adc-85d1-e7fa216f9a9c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:52.879394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401291-cb68-4adc-85d1-e7fa216f9a9c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:52.879496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f401291-cb68-4adc-85d1-e7fa216f9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:52.944339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4f401291-cb68-4adc-85d1-e7fa216f9a9c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:52.945556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4f401291-cb68-4adc-85d1-e7fa216f9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:53.044216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 656/1325] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:53.081731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:53.083378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:53.187378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84f776-08df-4cc6-8e5e-8c22e19ca524 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:53.261925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:53.262551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:53.262840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:53.263122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:53.264783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:53.284122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:53.381618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:53.383078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334067-0ffa-447e-be50-428498be604f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:53.401976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 670/1326] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:55.519040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:55.519598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:55.519598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:55.520959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 657/1327] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:55.526561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dfc60-0e6c-4f4e-b7cd-e34368e78a11 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:55.527031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dfc60-0e6c-4f4e-b7cd-e34368e78a11 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:55.527132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dfc60-0e6c-4f4e-b7cd-e34368e78a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:55.593029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7b9dfc60-0e6c-4f4e-b7cd-e34368e78a11 None None] MFA Rules not processed for user `5d213696814d4a9aaf5ee9a25ff03fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:55.594233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7b9dfc60-0e6c-4f4e-b7cd-e34368e78a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:55.668855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:55.715658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 671/1328] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:55.752713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:55.754427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:55.789508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:55.867174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9dde0-4071-49ce-b0fd-5ab25aa8935f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:55.941047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:55.941618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:55.941904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:55.942179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:55.943853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:56.030834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:56.064261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:56.065708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a456c-8e3c-4f85-afdf-121f69ed40b5 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:56.084937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 658/1329] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:44:58.398067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:58.398669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:58.398669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:58.399948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 672/1330] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:44:58.405224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c29e6-4f57-4d60-b858-662a8295d64e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:58.405630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c29e6-4f57-4d60-b858-662a8295d64e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:58.405790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c29e6-4f57-4d60-b858-662a8295d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:58.473368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-058c29e6-4f57-4d60-b858-662a8295d64e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:44:58.474602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-058c29e6-4f57-4d60-b858-662a8295d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:58.567626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 659/1331] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:44:58.603726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:44:58.605753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:58.709083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f25e9-cd5c-456f-97d5-e8573aa6f426 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:44:58.782814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:44:58.783454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:44:58.783736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:44:58.784014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:44:58.785649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:44:58.813009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:44:58.893224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:44:58.894686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a30c18-dd71-4920-bbb1-dc89c49df719 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:44:58.912024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 673/1332] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:01.001915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:01.002376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:01.002493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:01.003865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 660/1333] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:01.008930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c009-326c-44c3-b0c0-a8f9d78f05a5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:01.009370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c009-326c-44c3-b0c0-a8f9d78f05a5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:01.009500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c009-326c-44c3-b0c0-a8f9d78f05a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:01.073364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e019c009-326c-44c3-b0c0-a8f9d78f05a5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:01.074596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e019c009-326c-44c3-b0c0-a8f9d78f05a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:01.151375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:01.173545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 674/1334] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:01.213797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:01.215370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:01.254250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:01.317527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c247a93-31c7-47ff-97d9-3f4410eb4c68 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:01.388206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:01.388835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:01.389112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:01.389379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:01.391095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:01.471997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:01.497967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:01.499455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ead175-00f9-49d8-8415-263959f3f4db service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:01.516850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 661/1335] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:03.869392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:03.869917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:03.869917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:03.871277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 675/1336] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:03.876349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74934-3d58-4584-b8c1-dde1bab98ed0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:03.876733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74934-3d58-4584-b8c1-dde1bab98ed0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:03.876900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74934-3d58-4584-b8c1-dde1bab98ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:03.940881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c5c74934-3d58-4584-b8c1-dde1bab98ed0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:03.942074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c5c74934-3d58-4584-b8c1-dde1bab98ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:04.036415 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 662/1337] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:04.071956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:04.073974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:04.175506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d8452-1ab4-46ff-bf02-4705df3459b1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:04.247586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:04.248218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:04.248498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:04.248767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:04.250403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:04.285795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:04.358252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:04.359720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39ee94-103f-4d53-8ea5-5a138c73f6cb service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:04.379276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 676/1338] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:08.288523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:08.289082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:08.289082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:08.290488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 663/1339] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:08.295590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2fa1-86ee-4da0-a234-6ef5de778a6b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:08.296101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2fa1-86ee-4da0-a234-6ef5de778a6b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:08.296208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee2fa1-86ee-4da0-a234-6ef5de778a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:08.367317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-62ee2fa1-86ee-4da0-a234-6ef5de778a6b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:08.368526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-62ee2fa1-86ee-4da0-a234-6ef5de778a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:08.460214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:08.473580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 677/1340] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:08.513093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:08.515213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:08.564383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:08.610977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b04574-7da4-4c31-9d92-5ea37bea1294 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:08.614894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:08.615542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:08.615829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:08.616103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:08.626129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:08.627609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33794e36-30c2-4735-8ac8-e905d0ec9b77 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:08.637377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 664/1341] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:11.308059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f44eb-a268-4683-8793-e52072c4d3f3 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:11.308579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f44eb-a268-4683-8793-e52072c4d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:11.308691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f44eb-a268-4683-8793-e52072c4d3f3 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:11.310054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 678/1342] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:11.315581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952b3c-f97d-44e2-ba7e-a0d9fb868b8d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:11.315997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952b3c-f97d-44e2-ba7e-a0d9fb868b8d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:11.316133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7952b3c-f97d-44e2-ba7e-a0d9fb868b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:11.344001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c7952b3c-f97d-44e2-ba7e-a0d9fb868b8d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:11.345189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c7952b3c-f97d-44e2-ba7e-a0d9fb868b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:11.423342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 665/1343] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:11.461791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f44eb-a268-4683-8793-e52072c4d3f3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:11.473576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f44eb-a268-4683-8793-e52072c4d3f3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:11.477092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:11.477746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:11.478026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:11.478297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:11.488507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:11.489947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c87e0-627c-4316-91b3-48d3034d5ca5 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:11.501787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 679/1344] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:15.459472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80d4f-a116-4de7-a90c-b903a1f7e133 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:15.459998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80d4f-a116-4de7-a90c-b903a1f7e133 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:15.460114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c80d4f-a116-4de7-a90c-b903a1f7e133 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:15.461403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 666/1345] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:15.467503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3993e5-8b20-4fab-8b61-8c72c0e5727a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:15.467909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3993e5-8b20-4fab-8b61-8c72c0e5727a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:15.468069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3993e5-8b20-4fab-8b61-8c72c0e5727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:15.486140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6c3993e5-8b20-4fab-8b61-8c72c0e5727a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:15.487385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6c3993e5-8b20-4fab-8b61-8c72c0e5727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:15.495292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 680/1346] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:15.533217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c80d4f-a116-4de7-a90c-b903a1f7e133 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:15.544745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c80d4f-a116-4de7-a90c-b903a1f7e133 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:15.548340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:15.548973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:15.549255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:15.549533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:15.558760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:15.560654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48cf70-ed74-46d7-93de-cdd74e54ec2b service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:15.570043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 667/1347] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:20.578078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c21a2-9274-44b6-8642-a7f8d3d8ecc3 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:20.578936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c21a2-9274-44b6-8642-a7f8d3d8ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:20.579450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c21a2-9274-44b6-8642-a7f8d3d8ecc3 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:20.582308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 681/1348] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:20.587266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9bb32-bc66-4163-9a69-bf9bee002c99 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:20.588154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9bb32-bc66-4163-9a69-bf9bee002c99 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:20.588642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9bb32-bc66-4163-9a69-bf9bee002c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:20.678993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-92d9bb32-bc66-4163-9a69-bf9bee002c99 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:20.680523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-92d9bb32-bc66-4163-9a69-bf9bee002c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:20.733188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:20.814320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 668/1349] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:24.978491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6b64-8101-4d02-b507-8e4b12fc7b84 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:24.979094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6b64-8101-4d02-b507-8e4b12fc7b84 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:24.979094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6b64-8101-4d02-b507-8e4b12fc7b84 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:24.981184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 682/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:24.989466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9e719-5f65-4028-a511-7c89bcbbe3db None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:24.990435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9e719-5f65-4028-a511-7c89bcbbe3db None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:24.991555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9e719-5f65-4028-a511-7c89bcbbe3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:25.010626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-59a9e719-5f65-4028-a511-7c89bcbbe3db None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:25.011903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-59a9e719-5f65-4028-a511-7c89bcbbe3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:25.021003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 669/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:25.480966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=13 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:26.612675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec708b-fd79-48a3-bf37-e564c98b05c2 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:26.613413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec708b-fd79-48a3-bf37-e564c98b05c2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:26.613413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec708b-fd79-48a3-bf37-e564c98b05c2 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:26.614776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 683/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:26.619791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5993-c6a3-46d1-8859-9cb2f2569a9d None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:26.620221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5993-c6a3-46d1-8859-9cb2f2569a9d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:26.620349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5993-c6a3-46d1-8859-9cb2f2569a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:26.638686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e04f5993-c6a3-46d1-8859-9cb2f2569a9d None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:26.639885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e04f5993-c6a3-46d1-8859-9cb2f2569a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:26.648385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 670/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:45:30.198109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ac840-38a0-4eba-90ba-58332b995321 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:30.198657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ac840-38a0-4eba-90ba-58332b995321 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:30.198657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ac840-38a0-4eba-90ba-58332b995321 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:30.204799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 684/1354] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:30.206215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a5e-dc2c-47da-8b35-ec0ff4ad41d9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:30.206781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a5e-dc2c-47da-8b35-ec0ff4ad41d9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:30.207078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a5e-dc2c-47da-8b35-ec0ff4ad41d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:30.234246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6af35a5e-dc2c-47da-8b35-ec0ff4ad41d9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:30.236143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6af35a5e-dc2c-47da-8b35-ec0ff4ad41d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:30.246609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 671/1355] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:45:30.306241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de044c-5b6b-4a30-ac3d-a53396c3de34 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:30.306892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de044c-5b6b-4a30-ac3d-a53396c3de34 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:30.307340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de044c-5b6b-4a30-ac3d-a53396c3de34 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:30.309820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 685/1356] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 17:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:30.318095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2dd05-281a-467b-82d9-617db43a4ff9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:30.318484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2dd05-281a-467b-82d9-617db43a4ff9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:30.318692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2dd05-281a-467b-82d9-617db43a4ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:30.370233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c4f2dd05-281a-467b-82d9-617db43a4ff9 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:30.372059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c4f2dd05-281a-467b-82d9-617db43a4ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:30.458647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 672/1357] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 17:45:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:30.467324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de044c-5b6b-4a30-ac3d-a53396c3de34 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:30.483419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de044c-5b6b-4a30-ac3d-a53396c3de34 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:30.489395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:30.490246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:30.490869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:30.491346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:30.501119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:30.503120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fe1dd-11c5-4ca8-8e51-c8a245c9e1e4 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:30.514276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 686/1358] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:32.804502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631edb-71b9-4b5c-beec-c347e59020fd None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:32.805035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631edb-71b9-4b5c-beec-c347e59020fd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:32.805035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631edb-71b9-4b5c-beec-c347e59020fd None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:32.806603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 673/1359] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:32.812126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966ec1-75b1-489e-b9c0-4787a64ed167 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:32.812538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966ec1-75b1-489e-b9c0-4787a64ed167 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:32.812695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e966ec1-75b1-489e-b9c0-4787a64ed167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:32.831197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-1e966ec1-75b1-489e-b9c0-4787a64ed167 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:32.832413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1e966ec1-75b1-489e-b9c0-4787a64ed167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:32.840690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 687/1360] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:32.884877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631edb-71b9-4b5c-beec-c347e59020fd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:32.900223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631edb-71b9-4b5c-beec-c347e59020fd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:32.905808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:32.906554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:32.906909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:32.907257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:32.916019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:32.917616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b576f2-700a-4318-95f1-5b9a3344e7b3 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:32.927629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 674/1361] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:35.266445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105304-1388-45d8-914e-1f27660f66e7 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:35.266903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105304-1388-45d8-914e-1f27660f66e7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:35.266996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105304-1388-45d8-914e-1f27660f66e7 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:35.268361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 688/1362] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:35.273421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a8cc-fa59-4455-9182-51af599fe995 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:35.273827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a8cc-fa59-4455-9182-51af599fe995 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:35.273978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2a8cc-fa59-4455-9182-51af599fe995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:35.292340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e4f2a8cc-fa59-4455-9182-51af599fe995 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:35.294117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e4f2a8cc-fa59-4455-9182-51af599fe995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:35.302936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 675/1363] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:45:35.357830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b25d0-62a0-438f-9d2d-d50d832b1c81 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:35.358119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b25d0-62a0-438f-9d2d-d50d832b1c81 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:35.358600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b25d0-62a0-438f-9d2d-d50d832b1c81 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:35.360434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 689/1364] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 17:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:35.368561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a05-f3d5-45ca-a4fb-09c7a948745a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:35.368561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a05-f3d5-45ca-a4fb-09c7a948745a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:35.368561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a05-f3d5-45ca-a4fb-09c7a948745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:35.393034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3f343a05-f3d5-45ca-a4fb-09c7a948745a None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:35.394235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3f343a05-f3d5-45ca-a4fb-09c7a948745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:35.404110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 676/1365] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 17:45:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:35.411971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b25d0-62a0-438f-9d2d-d50d832b1c81 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:35.426031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b25d0-62a0-438f-9d2d-d50d832b1c81 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:35.431288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:35.431909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:35.432196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:35.432474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:35.440830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:35.442426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9b592-2fb5-450e-8642-f4207e4c02a5 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:35.452223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 690/1366] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:37.551890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c6018-0f5c-494b-ab8f-c48529a29b5c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:37.552442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c6018-0f5c-494b-ab8f-c48529a29b5c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:37.552442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c6018-0f5c-494b-ab8f-c48529a29b5c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:37.553810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 677/1367] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:37.559673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f438e-d182-40a7-afad-e3c5e1fb27e3 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:37.560418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f438e-d182-40a7-afad-e3c5e1fb27e3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:37.560701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f438e-d182-40a7-afad-e3c5e1fb27e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:37.579626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-280f438e-d182-40a7-afad-e3c5e1fb27e3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:37.580835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-280f438e-d182-40a7-afad-e3c5e1fb27e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:37.588814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 691/1368] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:37.632869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c6018-0f5c-494b-ab8f-c48529a29b5c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:37.647572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c6018-0f5c-494b-ab8f-c48529a29b5c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:37.653217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:37.653997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:37.654373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:37.654725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:37.664784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:37.666543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b419ae-1a18-4d8f-9df2-9f9cb248bf6c service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:37.678598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 678/1369] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:38.355941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71028-1fb5-43b6-90bf-cd83276b4036 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:38.356349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71028-1fb5-43b6-90bf-cd83276b4036 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:38.356669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f71028-1fb5-43b6-90bf-cd83276b4036 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:38.358247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 692/1370] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:38.363195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f220d-ab62-4567-a14d-54e0e8163fb7 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:38.363491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f220d-ab62-4567-a14d-54e0e8163fb7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:38.363770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f220d-ab62-4567-a14d-54e0e8163fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:38.381716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5f9f220d-ab62-4567-a14d-54e0e8163fb7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:38.382960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5f9f220d-ab62-4567-a14d-54e0e8163fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:38.392280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 679/1371] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:38.741579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44fe1b-753a-4bc7-b35a-cc194b491e1c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:38.741579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44fe1b-753a-4bc7-b35a-cc194b491e1c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:38.741579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44fe1b-753a-4bc7-b35a-cc194b491e1c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:38.743532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 693/1372] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jun 30 17:45:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:38.750734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896701-e5cb-4ab6-9024-1baa0a043d09 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:38.751144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896701-e5cb-4ab6-9024-1baa0a043d09 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:38.751296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896701-e5cb-4ab6-9024-1baa0a043d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:38.759222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:38.804893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6f896701-e5cb-4ab6-9024-1baa0a043d09 None None] MFA Rules not processed for user `cd5dd586ede14a918301fd93b4c8f320`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:38.806639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6f896701-e5cb-4ab6-9024-1baa0a043d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:38.874418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 680/1373] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jun 30 17:45:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:38.882422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44fe1b-753a-4bc7-b35a-cc194b491e1c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:38.893712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44fe1b-753a-4bc7-b35a-cc194b491e1c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:38.898242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:38.898983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:38.899485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:38.899829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:38.909771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:38.911311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0e76-482d-412e-9f0b-696db1328b5f service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:38.921847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 694/1374] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:39.754930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9857-519e-40bf-86b2-3c4c95689319 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:39.754930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9857-519e-40bf-86b2-3c4c95689319 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:39.754930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9857-519e-40bf-86b2-3c4c95689319 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:39.757782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 681/1375] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:39.763977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a9c1-2751-4b2a-9a7a-fae1dae6cad2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:39.764355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a9c1-2751-4b2a-9a7a-fae1dae6cad2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:39.765075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a9c1-2751-4b2a-9a7a-fae1dae6cad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:39.784663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a530a9c1-2751-4b2a-9a7a-fae1dae6cad2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:39.786082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a530a9c1-2751-4b2a-9a7a-fae1dae6cad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:39.794962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 695/1376] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:39.840051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9857-519e-40bf-86b2-3c4c95689319 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:39.854614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9857-519e-40bf-86b2-3c4c95689319 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:39.860689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:39.861394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:39.861749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:39.862095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:39.870250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:39.872525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2640cca-3383-4255-9fa5-ac914493299a service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:39.882100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 682/1377] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:41.943053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b39aa-de30-405e-ba13-cc0ce92723c2 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:41.943587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b39aa-de30-405e-ba13-cc0ce92723c2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:41.943848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b39aa-de30-405e-ba13-cc0ce92723c2 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:41.945508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 696/1378] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:41.951531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57065-3ebd-4eb0-b6e2-20d30774541b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:41.951942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57065-3ebd-4eb0-b6e2-20d30774541b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:41.952073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57065-3ebd-4eb0-b6e2-20d30774541b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:41.978045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-7be57065-3ebd-4eb0-b6e2-20d30774541b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:41.979604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7be57065-3ebd-4eb0-b6e2-20d30774541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:41.994875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 683/1379] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:42.043229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b39aa-de30-405e-ba13-cc0ce92723c2 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:42.057819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b39aa-de30-405e-ba13-cc0ce92723c2 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:42.063853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:42.064750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:42.065035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:42.065314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:42.073439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:42.075810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdb4d2-3001-4f60-bf58-55ab138ae144 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:42.085521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 697/1380] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:44.227068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd20b0-833b-4da4-87f1-b460fad80027 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:44.227556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd20b0-833b-4da4-87f1-b460fad80027 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:44.227656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd20b0-833b-4da4-87f1-b460fad80027 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:44.229579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 684/1381] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:44.234017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251912-b3f8-40bc-a30c-79050924b159 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:44.234465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251912-b3f8-40bc-a30c-79050924b159 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:44.234620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251912-b3f8-40bc-a30c-79050924b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:44.252968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f7251912-b3f8-40bc-a30c-79050924b159 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:44.254077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f7251912-b3f8-40bc-a30c-79050924b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:44.262761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 698/1382] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:44.304594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd20b0-833b-4da4-87f1-b460fad80027 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:44.319695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd20b0-833b-4da4-87f1-b460fad80027 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:44.325779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:44.326439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:44.326721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:44.326990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:44.335948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:44.337510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2a164-65b3-47b2-9e99-e069595b0f0e service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:44.347438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 685/1383] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:46.434643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439499af-451e-45d0-af2c-04d8852cba06 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:46.434643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439499af-451e-45d0-af2c-04d8852cba06 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:46.434643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439499af-451e-45d0-af2c-04d8852cba06 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:46.438249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 699/1384] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:46.448859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4212-50a4-4121-bbb2-d4ecd8622da6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:46.449419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4212-50a4-4121-bbb2-d4ecd8622da6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:46.449769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa4212-50a4-4121-bbb2-d4ecd8622da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:46.487158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-02aa4212-50a4-4121-bbb2-d4ecd8622da6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:46.488373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-02aa4212-50a4-4121-bbb2-d4ecd8622da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:46.497562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 686/1385] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:46.548405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439499af-451e-45d0-af2c-04d8852cba06 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:46.562893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439499af-451e-45d0-af2c-04d8852cba06 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:46.568905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:46.569552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:46.569838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:46.570116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:46.579312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:46.580910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98cbf69-9089-440f-be61-ffc3221e7835 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:46.591442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 700/1386] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:48.690785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b5d0-03ef-4fc2-9e36-827590a1a041 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:48.691343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b5d0-03ef-4fc2-9e36-827590a1a041 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:48.691343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5b5d0-03ef-4fc2-9e36-827590a1a041 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:48.692720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 687/1387] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:48.698070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7483f-cc4b-4092-91a6-9301c4a63524 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:48.698572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7483f-cc4b-4092-91a6-9301c4a63524 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:48.698693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7483f-cc4b-4092-91a6-9301c4a63524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:48.720886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-98b7483f-cc4b-4092-91a6-9301c4a63524 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:48.723120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-98b7483f-cc4b-4092-91a6-9301c4a63524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:48.733514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 701/1388] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:48.781769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b5d0-03ef-4fc2-9e36-827590a1a041 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:48.798407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5b5d0-03ef-4fc2-9e36-827590a1a041 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:48.803856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:48.804494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:48.804778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:48.805050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:48.814404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:48.816250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9aaed-568c-442d-8d3c-3a79d4dfddfc service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:48.826467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 688/1389] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:50.914868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a05e-7bb4-4784-bfcd-2d4a4381e5d7 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:50.915551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a05e-7bb4-4784-bfcd-2d4a4381e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:50.915551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32a05e-7bb4-4784-bfcd-2d4a4381e5d7 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:50.916834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 702/1390] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:50.923365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b22b-8e66-4ba7-952d-715d2d073e7c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:50.923912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b22b-8e66-4ba7-952d-715d2d073e7c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:50.924077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6b22b-8e66-4ba7-952d-715d2d073e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:50.944917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-91d6b22b-8e66-4ba7-952d-715d2d073e7c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:50.946156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-91d6b22b-8e66-4ba7-952d-715d2d073e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:50.954278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 689/1391] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:45:50.999231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a05e-7bb4-4784-bfcd-2d4a4381e5d7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:51.013650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32a05e-7bb4-4784-bfcd-2d4a4381e5d7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:51.019331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:51.019974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:51.020260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:51.020545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:51.029737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:51.031356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b677325-ba1b-49eb-b87d-51a4f27efffb service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:51.041146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 703/1392] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:53.131335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e3c6-a145-44ca-8d10-ca6668a6923a None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:53.131891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e3c6-a145-44ca-8d10-ca6668a6923a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:53.131891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964e3c6-a145-44ca-8d10-ca6668a6923a None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:53.133420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 690/1393] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:53.139163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca221b2-8346-409b-a5e1-ed13c18db6a8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:53.139583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca221b2-8346-409b-a5e1-ed13c18db6a8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:53.139723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca221b2-8346-409b-a5e1-ed13c18db6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:53.157098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fca221b2-8346-409b-a5e1-ed13c18db6a8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:53.158284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fca221b2-8346-409b-a5e1-ed13c18db6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:53.165959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 704/1394] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:53.210675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964e3c6-a145-44ca-8d10-ca6668a6923a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:53.224831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964e3c6-a145-44ca-8d10-ca6668a6923a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:53.230135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:53.230800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:53.231103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:53.231382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:53.239424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:53.241088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bd205-7780-4a0a-a079-a08e6f3dcb2b service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:53.251178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 691/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:54.330745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7ce2-aed3-4257-82ad-bed45ef157ad None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:54.331390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7ce2-aed3-4257-82ad-bed45ef157ad None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:54.331390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7ce2-aed3-4257-82ad-bed45ef157ad None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:54.332737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 705/1396] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:54.338098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605d69f-f991-4987-835d-ae1632b42c49 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:54.338935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605d69f-f991-4987-835d-ae1632b42c49 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:54.339220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605d69f-f991-4987-835d-ae1632b42c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:54.357377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e605d69f-f991-4987-835d-ae1632b42c49 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:54.358605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e605d69f-f991-4987-835d-ae1632b42c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:54.366377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 692/1397] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:54.422531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440e9bf-82bd-4949-91db-6808fbb02872 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:54.422827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440e9bf-82bd-4949-91db-6808fbb02872 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:54.423102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440e9bf-82bd-4949-91db-6808fbb02872 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:54.424461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 706/1398] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jun 30 17:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:54.429362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08d29-8c7a-4774-acbc-6b8eb43af25c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:54.429658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08d29-8c7a-4774-acbc-6b8eb43af25c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:54.429938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08d29-8c7a-4774-acbc-6b8eb43af25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:54.480976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4dc08d29-8c7a-4774-acbc-6b8eb43af25c None None] MFA Rules not processed for user `2861607f39d44172851da17c72704358`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:54.482241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4dc08d29-8c7a-4774-acbc-6b8eb43af25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:54.515706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:54.544325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 693/1399] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jun 30 17:45:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:54.551777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440e9bf-82bd-4949-91db-6808fbb02872 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:54.562970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440e9bf-82bd-4949-91db-6808fbb02872 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:54.566733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:54.567368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:54.567654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:54.567933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:54.577485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:54.578965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031e9d3-bb63-418f-ac1a-2c5cbc284495 service glance] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:54.588693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 707/1400] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:45:55.300093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e61b-e18c-4487-baaf-8fcb57359e7e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.300381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e61b-e18c-4487-baaf-8fcb57359e7e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.300658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e61b-e18c-4487-baaf-8fcb57359e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.335491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53cf3-d197-4801-90b6-213c782feb70 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.336047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53cf3-d197-4801-90b6-213c782feb70 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.336047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53cf3-d197-4801-90b6-213c782feb70 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.337412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 708/1401] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:45:55.343552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.343844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.344123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.353583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1445e61b-e18c-4487-baaf-8fcb57359e7e None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:55.354865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1445e61b-e18c-4487-baaf-8fcb57359e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:55.362619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:55.363790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:55.371595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 709/1402] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:45:55.415661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:55.420413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 694/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:55.428830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a238b-83dd-4986-83bf-8f882d497ec3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:55.435056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:55.435713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.435996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.436276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.446701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:55.448309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01853ac-98c9-4b6a-a0d2-baef0f236723 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:55.458590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 710/1404] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:45:55.471115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d83e2-da22-4984-abd5-343953d2cbe0 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.471506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d83e2-da22-4984-abd5-343953d2cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.472006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d83e2-da22-4984-abd5-343953d2cbe0 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.474075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 695/1405] 10.222.0.16 () {54 vars in 942 bytes} [Thu Jun 30 17:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:45:55.480854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a261e5-356f-4d3f-988d-feb286ac44bb None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.481145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a261e5-356f-4d3f-988d-feb286ac44bb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.482072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a261e5-356f-4d3f-988d-feb286ac44bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.499652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=73 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:45:55.538034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-44a261e5-356f-4d3f-988d-feb286ac44bb None None] MFA Rules not processed for user `3ec0b4d695474375b2f87727d41e40f7`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:45:55.539311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-44a261e5-356f-4d3f-988d-feb286ac44bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:45:55.620190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 711/1406] 10.222.0.16 () {58 vars in 1039 bytes} [Thu Jun 30 17:45:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:45:55.629928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d83e2-da22-4984-abd5-343953d2cbe0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:45:55.644607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d83e2-da22-4984-abd5-343953d2cbe0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:45:55.649923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:45:55.650605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:45:55.650890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:45:55.651165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:45:55.660997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:45:55.662688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51923ffc-7dc3-4e20-9b7b-e47e24d0c8bc service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:45:55.671646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 696/1407] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 17:45:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 17:46:05.616996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=40 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:46:05.668108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=56 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:46:18.724428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a52f-af87-4a10-8688-2c245478615f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:18.724998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a52f-af87-4a10-8688-2c245478615f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:18.724998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a52f-af87-4a10-8688-2c245478615f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:18.726479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 712/1408] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jun 30 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:18.731386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836fd20-ec33-4f3d-8027-cbd836f38c2b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:18.731867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836fd20-ec33-4f3d-8027-cbd836f38c2b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:18.731973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836fd20-ec33-4f3d-8027-cbd836f38c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:18.750036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f836fd20-ec33-4f3d-8027-cbd836f38c2b None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:18.751270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f836fd20-ec33-4f3d-8027-cbd836f38c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:18.759571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 697/1409] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 30 17:46:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:18.773238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431157-b0d8-4b37-974c-e67f2fd6f020 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:18.773532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431157-b0d8-4b37-974c-e67f2fd6f020 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:18.773814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431157-b0d8-4b37-974c-e67f2fd6f020 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:18.775188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 713/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jun 30 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:18.780246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffaaba-a98e-4f41-8691-f41c9d9aa960 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:18.780531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffaaba-a98e-4f41-8691-f41c9d9aa960 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:18.780807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffaaba-a98e-4f41-8691-f41c9d9aa960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:18.799436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9dffaaba-a98e-4f41-8691-f41c9d9aa960 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:18.800783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9dffaaba-a98e-4f41-8691-f41c9d9aa960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:18.809774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 698/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 30 17:46:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:46:21.298645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba2bcd-a49f-47f4-a888-ae19d081ddbf None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:21.299171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba2bcd-a49f-47f4-a888-ae19d081ddbf None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:21.299388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba2bcd-a49f-47f4-a888-ae19d081ddbf None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:21.301338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 714/1412] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:21.306499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0ec5-6f35-4fed-bcf1-0a6f89a2fb2e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:21.306922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0ec5-6f35-4fed-bcf1-0a6f89a2fb2e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:21.307063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0ec5-6f35-4fed-bcf1-0a6f89a2fb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:21.325001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-30bc0ec5-6f35-4fed-bcf1-0a6f89a2fb2e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:21.326217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-30bc0ec5-6f35-4fed-bcf1-0a6f89a2fb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:21.333974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 699/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:21.377335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba2bcd-a49f-47f4-a888-ae19d081ddbf None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:21.392186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba2bcd-a49f-47f4-a888-ae19d081ddbf None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:21.397552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:21.398183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:21.398506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:21.398786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:21.406850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:21.408413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc315b5c-743c-4bc7-a503-b4a1d7769a28 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:21.417637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 715/1414] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:22.318357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d704a5-a731-4b34-9237-652352d0e999 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:22.318902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d704a5-a731-4b34-9237-652352d0e999 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:22.318902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d704a5-a731-4b34-9237-652352d0e999 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:22.320303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 700/1415] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:22.325190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731369c3-d0c4-4a9d-8444-f8f456735d25 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:22.325604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731369c3-d0c4-4a9d-8444-f8f456735d25 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:22.325799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731369c3-d0c4-4a9d-8444-f8f456735d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:22.344173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-731369c3-d0c4-4a9d-8444-f8f456735d25 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:22.345367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-731369c3-d0c4-4a9d-8444-f8f456735d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:22.352978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 716/1416] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:22.502926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d704a5-a731-4b34-9237-652352d0e999 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:22.517428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d704a5-a731-4b34-9237-652352d0e999 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:22.522680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:22.523306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:22.523593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:22.523865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:22.531814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:22.533366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b91308-7c62-4832-9f12-46c054fc5d55 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:22.542557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 701/1417] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:23.417328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdcf86-6751-45bc-8fd5-edcd1370ffd5 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:23.417870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdcf86-6751-45bc-8fd5-edcd1370ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:23.417870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdcf86-6751-45bc-8fd5-edcd1370ffd5 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:23.419275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 717/1418] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:23.424200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5e57-2cf9-4066-a80a-b076e1a4f23e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:23.424575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5e57-2cf9-4066-a80a-b076e1a4f23e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:23.424765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d5e57-2cf9-4066-a80a-b076e1a4f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:23.442814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-876d5e57-2cf9-4066-a80a-b076e1a4f23e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:23.444071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-876d5e57-2cf9-4066-a80a-b076e1a4f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:23.452765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 702/1419] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:23.605719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdcf86-6751-45bc-8fd5-edcd1370ffd5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:23.620291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdcf86-6751-45bc-8fd5-edcd1370ffd5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:23.625656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:23.626293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:23.626608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:23.626885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:23.634967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:23.636564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c471c-aee9-4bda-a0f3-2220684929d7 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:23.646496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 718/1420] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:24.510258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ca86-668e-4c6f-a62c-e8bdc5c608dc None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:24.510822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ca86-668e-4c6f-a62c-e8bdc5c608dc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:24.510822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20ca86-668e-4c6f-a62c-e8bdc5c608dc None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:24.512203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 703/1421] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:24.517302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0d9-25c8-44ca-b66c-6383b208c310 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:24.517681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0d9-25c8-44ca-b66c-6383b208c310 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:24.517865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0d9-25c8-44ca-b66c-6383b208c310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:24.536025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a065b0d9-25c8-44ca-b66c-6383b208c310 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:24.537211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a065b0d9-25c8-44ca-b66c-6383b208c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:24.545525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 719/1422] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:24.698804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20ca86-668e-4c6f-a62c-e8bdc5c608dc None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:24.712914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20ca86-668e-4c6f-a62c-e8bdc5c608dc None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:24.718837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:24.719469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:24.719755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:24.720033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:24.728751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:24.730353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c5b87-ac64-486c-8b75-2e65f930087e service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:24.740301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 704/1423] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:25.601779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc720e8-eb25-49d8-ad78-ebb6e57e46ca None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:25.602365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc720e8-eb25-49d8-ad78-ebb6e57e46ca None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:25.602365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc720e8-eb25-49d8-ad78-ebb6e57e46ca None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:25.603744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 720/1424] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:25.608719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292baf-34dc-4376-b9d3-973a941cbf0a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:25.609107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292baf-34dc-4376-b9d3-973a941cbf0a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:25.609274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292baf-34dc-4376-b9d3-973a941cbf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:25.627124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-46292baf-34dc-4376-b9d3-973a941cbf0a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:25.628327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-46292baf-34dc-4376-b9d3-973a941cbf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:25.636265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 705/1425] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:25.788526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc720e8-eb25-49d8-ad78-ebb6e57e46ca None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:25.802889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc720e8-eb25-49d8-ad78-ebb6e57e46ca None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:25.808494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:25.809126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:25.809434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:25.809709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:25.818303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:25.819902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b6556-9b72-4ad4-a3f6-b943084fe7f8 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:25.829857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 721/1426] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:26.636170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7743-0ab9-4c63-be39-da10351a70e6 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:26.636739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7743-0ab9-4c63-be39-da10351a70e6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:26.636739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7743-0ab9-4c63-be39-da10351a70e6 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:26.638080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 706/1427] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:26.642902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab2c0-b3ec-4cc8-967d-da7deaeabebd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:26.643298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab2c0-b3ec-4cc8-967d-da7deaeabebd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:26.643459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab2c0-b3ec-4cc8-967d-da7deaeabebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:26.661042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b25ab2c0-b3ec-4cc8-967d-da7deaeabebd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:26.662237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b25ab2c0-b3ec-4cc8-967d-da7deaeabebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:26.670458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 722/1428] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:26.830211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7743-0ab9-4c63-be39-da10351a70e6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:26.844822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7743-0ab9-4c63-be39-da10351a70e6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:26.850140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:26.850808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:26.851094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:26.851373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:26.860061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:26.861637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ed1d-6ea4-480f-a211-631711829a68 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:26.871899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 707/1429] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:27.675650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690f695-e70e-4e5a-9580-6efc763fed7a None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:27.676178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690f695-e70e-4e5a-9580-6efc763fed7a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:27.676278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690f695-e70e-4e5a-9580-6efc763fed7a None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:27.677578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 723/1430] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:27.682363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da723-ff79-472a-9b75-f870bd841508 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:27.682768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da723-ff79-472a-9b75-f870bd841508 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:27.682929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2da723-ff79-472a-9b75-f870bd841508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:27.700649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6e2da723-ff79-472a-9b75-f870bd841508 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:27.701871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6e2da723-ff79-472a-9b75-f870bd841508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:27.709558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 708/1431] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:27.858807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690f695-e70e-4e5a-9580-6efc763fed7a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:27.873268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690f695-e70e-4e5a-9580-6efc763fed7a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:27.878565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:27.879188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:27.879467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:27.879737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:27.887611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:27.890194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87852e-c404-45fc-96e6-996cda32474d service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:27.899757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 724/1432] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:28.704234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfb169-ba0f-4d2d-83c5-0bbf0fff0b5f None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:28.704804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfb169-ba0f-4d2d-83c5-0bbf0fff0b5f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:28.705031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfb169-ba0f-4d2d-83c5-0bbf0fff0b5f None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:28.706795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 709/1433] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:28.711836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4233e3-90fc-4dea-8037-5b5e5c8437f2 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:28.712282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4233e3-90fc-4dea-8037-5b5e5c8437f2 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:28.712389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4233e3-90fc-4dea-8037-5b5e5c8437f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:28.731593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7d4233e3-90fc-4dea-8037-5b5e5c8437f2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:28.732778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7d4233e3-90fc-4dea-8037-5b5e5c8437f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:28.740670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 725/1434] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:28.891395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfb169-ba0f-4d2d-83c5-0bbf0fff0b5f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:28.906366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfb169-ba0f-4d2d-83c5-0bbf0fff0b5f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:28.911621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:28.912266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:28.912546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:28.912820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:28.922016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:28.923663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917ba0f-cab2-46b9-b064-f26e94ea0d0f service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:28.933177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 710/1435] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:29.723530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec971-ccad-4f34-bbc3-2cb5148ac418 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:29.724081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec971-ccad-4f34-bbc3-2cb5148ac418 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:29.724180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aec971-ccad-4f34-bbc3-2cb5148ac418 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:29.725868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 726/1436] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:29.730606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb67095-7961-4e89-ab8d-a6b459740c80 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:29.731048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb67095-7961-4e89-ab8d-a6b459740c80 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:29.731198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb67095-7961-4e89-ab8d-a6b459740c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:29.749444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0eb67095-7961-4e89-ab8d-a6b459740c80 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:29.754298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0eb67095-7961-4e89-ab8d-a6b459740c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:29.763055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 711/1437] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:29.914783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aec971-ccad-4f34-bbc3-2cb5148ac418 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:29.928911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aec971-ccad-4f34-bbc3-2cb5148ac418 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:29.934228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:29.934905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:29.935189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:29.935469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:29.945579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:29.947203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c59170-b70b-405e-a995-5b818bca1112 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:29.957757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 727/1438] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:31.905341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9ad50-f046-429b-aadd-8878b7df22cd None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:31.905937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9ad50-f046-429b-aadd-8878b7df22cd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:31.905937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9ad50-f046-429b-aadd-8878b7df22cd None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:31.907280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 712/1439] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:31.912411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbf89-5a33-4fea-b295-ad17c427261a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:31.912849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbf89-5a33-4fea-b295-ad17c427261a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:31.912976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbf89-5a33-4fea-b295-ad17c427261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:31.930719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d71cbf89-5a33-4fea-b295-ad17c427261a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:31.931912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d71cbf89-5a33-4fea-b295-ad17c427261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:31.940106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 728/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:31.978516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9ad50-f046-429b-aadd-8878b7df22cd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:31.989843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9ad50-f046-429b-aadd-8878b7df22cd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:31.993399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:31.994029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:31.994310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:31.994610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:32.003553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:32.004971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e88623-41d5-46ed-991c-66d66c752778 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:32.014201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 713/1441] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:34.199263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85166-d322-42af-9700-df3e8a2a3718 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:34.199813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85166-d322-42af-9700-df3e8a2a3718 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:34.199813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f85166-d322-42af-9700-df3e8a2a3718 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:34.201178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 729/1442] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:34.207078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c899f-20a0-445e-8f82-4264e02247e4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:34.207508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c899f-20a0-445e-8f82-4264e02247e4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:34.207663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c899f-20a0-445e-8f82-4264e02247e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:34.225063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-064c899f-20a0-445e-8f82-4264e02247e4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:34.226261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-064c899f-20a0-445e-8f82-4264e02247e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:34.234032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 714/1443] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:34.273115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f85166-d322-42af-9700-df3e8a2a3718 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:34.283608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f85166-d322-42af-9700-df3e8a2a3718 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:34.287188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:34.287815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:34.288100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:34.288375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:34.297030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:34.298476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c351e8-0c8d-4ab7-bb85-6318759b6081 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:34.307681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 730/1444] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:36.659132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bbbbf-1621-4816-a2f8-c53330f1de86 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:36.659660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bbbbf-1621-4816-a2f8-c53330f1de86 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:36.659660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bbbbf-1621-4816-a2f8-c53330f1de86 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:36.660982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 715/1445] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:36.666264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d0fd0-6f6b-4e4b-9adb-cc14395fa6e7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:36.666710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d0fd0-6f6b-4e4b-9adb-cc14395fa6e7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:36.666890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d0fd0-6f6b-4e4b-9adb-cc14395fa6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:36.684764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ec0d0fd0-6f6b-4e4b-9adb-cc14395fa6e7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:36.685991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ec0d0fd0-6f6b-4e4b-9adb-cc14395fa6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:36.694028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 731/1446] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:36.733602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bbbbf-1621-4816-a2f8-c53330f1de86 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:36.744239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bbbbf-1621-4816-a2f8-c53330f1de86 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:36.747731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:36.748356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:36.748635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:36.748907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:36.757841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:36.759369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55981476-f81e-4569-a5b0-f92a1fc0b87f service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:36.768637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 716/1447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:38.969551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641745f5-44b6-48be-8b15-aa7b4f30ed53 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:38.970106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641745f5-44b6-48be-8b15-aa7b4f30ed53 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:38.970106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641745f5-44b6-48be-8b15-aa7b4f30ed53 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:38.971503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 732/1448] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:38.976638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590337-31d4-45d3-9d22-c7f8eeb4f674 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:38.977076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590337-31d4-45d3-9d22-c7f8eeb4f674 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:38.977220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590337-31d4-45d3-9d22-c7f8eeb4f674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:38.996571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b5590337-31d4-45d3-9d22-c7f8eeb4f674 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:38.997800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b5590337-31d4-45d3-9d22-c7f8eeb4f674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:39.006181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 717/1449] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:46:39.044613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641745f5-44b6-48be-8b15-aa7b4f30ed53 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:39.055205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641745f5-44b6-48be-8b15-aa7b4f30ed53 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:39.058690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:39.059318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:39.059601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:39.059876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:39.069354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:39.070840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef6fea-0034-49ac-b38f-492e59a00a2f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:39.080631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 733/1450] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:41.155789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2944b-1e29-4a7c-aaaf-73a482017316 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:41.156347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2944b-1e29-4a7c-aaaf-73a482017316 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:41.156347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2944b-1e29-4a7c-aaaf-73a482017316 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:41.157691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 718/1451] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:41.163222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e577235-08cc-43af-b79b-4451ac5256a8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:41.163624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e577235-08cc-43af-b79b-4451ac5256a8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:41.163785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e577235-08cc-43af-b79b-4451ac5256a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:41.180959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8e577235-08cc-43af-b79b-4451ac5256a8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:41.182150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8e577235-08cc-43af-b79b-4451ac5256a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:41.190252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 734/1452] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:41.227830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2944b-1e29-4a7c-aaaf-73a482017316 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:41.238455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2944b-1e29-4a7c-aaaf-73a482017316 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:41.241935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:41.242528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:41.242809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:41.243080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:41.252792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:41.254209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b09d3-1366-42da-94e5-30929c1f9f35 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:41.263368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 719/1453] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:43.542556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5159a-1403-492a-b58c-1f8bd555bdab None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:43.543095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5159a-1403-492a-b58c-1f8bd555bdab None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:43.543219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5159a-1403-492a-b58c-1f8bd555bdab None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:43.544511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 735/1454] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:43.550445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6c909-d1de-4e61-a2e5-fb6e24c83a9e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:43.550847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6c909-d1de-4e61-a2e5-fb6e24c83a9e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:43.551008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6c909-d1de-4e61-a2e5-fb6e24c83a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:43.568685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-22b6c909-d1de-4e61-a2e5-fb6e24c83a9e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:43.569881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-22b6c909-d1de-4e61-a2e5-fb6e24c83a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:43.578158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 720/1455] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:43.616088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5159a-1403-492a-b58c-1f8bd555bdab None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:43.626846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5159a-1403-492a-b58c-1f8bd555bdab None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:43.630503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:43.631141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:43.631459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:43.631738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:43.641124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:43.642591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34c246-6f7d-4055-a561-3daad988e85b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:43.651900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 736/1456] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:45.721768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162deaa-1556-4ced-9bf7-4c7641d284c5 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:45.722303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162deaa-1556-4ced-9bf7-4c7641d284c5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:45.722303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162deaa-1556-4ced-9bf7-4c7641d284c5 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:45.723713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 721/1457] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:45.729221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fad31f-1083-45c2-9d66-c84d004f411a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:45.729669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fad31f-1083-45c2-9d66-c84d004f411a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:45.729774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fad31f-1083-45c2-9d66-c84d004f411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:45.736770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=70 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:46:45.748966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c8fad31f-1083-45c2-9d66-c84d004f411a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:45.750166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c8fad31f-1083-45c2-9d66-c84d004f411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:45.757879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 737/1458] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:45.795797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162deaa-1556-4ced-9bf7-4c7641d284c5 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:45.801652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=73 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:46:45.807916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162deaa-1556-4ced-9bf7-4c7641d284c5 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:45.811372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:45.812003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:45.812549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:45.812670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:45.821629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:45.823091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca12c09-3ca0-4932-bc14-fc2d51a33a1c service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:45.832790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 722/1459] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:47.908117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408306c7-5534-4670-85c2-cadd88923f2c None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:47.908641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408306c7-5534-4670-85c2-cadd88923f2c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:47.908641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408306c7-5534-4670-85c2-cadd88923f2c None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:47.909942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 738/1460] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:47.915086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48227a7-53b5-4318-9194-4560026c323e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:47.915507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48227a7-53b5-4318-9194-4560026c323e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:47.915681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48227a7-53b5-4318-9194-4560026c323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:47.933598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f48227a7-53b5-4318-9194-4560026c323e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:47.934820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f48227a7-53b5-4318-9194-4560026c323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:47.942538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 723/1461] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:47.979767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408306c7-5534-4670-85c2-cadd88923f2c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:47.993240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408306c7-5534-4670-85c2-cadd88923f2c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:47.996723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:47.997359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:47.997639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:47.997914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:48.006965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:48.008442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8f4a0-4b65-47da-b28f-d78eac16295e service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:48.017828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 739/1462] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:50.068123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d93ea-48fe-434e-8d93-523b952fac70 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:50.068915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d93ea-48fe-434e-8d93-523b952fac70 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:50.068915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d93ea-48fe-434e-8d93-523b952fac70 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:50.070397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 724/1463] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:50.075745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2615-bf5e-4487-b8e5-59709dabc0d6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:50.076173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2615-bf5e-4487-b8e5-59709dabc0d6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:50.076309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da2615-bf5e-4487-b8e5-59709dabc0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:50.095122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-47da2615-bf5e-4487-b8e5-59709dabc0d6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:50.096329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-47da2615-bf5e-4487-b8e5-59709dabc0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:50.104105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 740/1464] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:50.140370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d93ea-48fe-434e-8d93-523b952fac70 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:50.151695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d93ea-48fe-434e-8d93-523b952fac70 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:50.155249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:50.155876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:50.156159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:50.156435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:50.165355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:50.166818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ffa09-3507-4097-a330-462765fa8c3e service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:50.176021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 725/1465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:46:52.207361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab1e-29a0-4996-b05b-bb5f2393ca60 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:52.207889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab1e-29a0-4996-b05b-bb5f2393ca60 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:52.207889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab1e-29a0-4996-b05b-bb5f2393ca60 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:52.209270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 741/1466] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:46:52.214540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41691694-b169-44be-aad3-eacf48a8b5fb None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:52.214951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41691694-b169-44be-aad3-eacf48a8b5fb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:52.215107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41691694-b169-44be-aad3-eacf48a8b5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:52.233428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-41691694-b169-44be-aad3-eacf48a8b5fb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:46:52.235258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-41691694-b169-44be-aad3-eacf48a8b5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:46:52.242950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 726/1467] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:46:52.279569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ab1e-29a0-4996-b05b-bb5f2393ca60 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:46:52.290567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ab1e-29a0-4996-b05b-bb5f2393ca60 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:46:52.293955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:46:52.294615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:46:52.294896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:46:52.295166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:46:52.304430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:46:52.305834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebef41-c7f5-403f-b8f8-a638787c5d3d service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:46:52.314957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 742/1468] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:47:02.227899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=17 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:47:02.310910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=17 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:47:12.076900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45cc1-ee85-4d3c-9ada-bbf1e545a31c None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:12.077480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45cc1-ee85-4d3c-9ada-bbf1e545a31c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:12.077480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af45cc1-ee85-4d3c-9ada-bbf1e545a31c None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:12.079398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 727/1469] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:12.084688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c77c9-e96f-4d1d-80bd-717a7893489b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:12.085125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c77c9-e96f-4d1d-80bd-717a7893489b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:12.085225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c77c9-e96f-4d1d-80bd-717a7893489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:12.102981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-476c77c9-e96f-4d1d-80bd-717a7893489b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:12.104163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-476c77c9-e96f-4d1d-80bd-717a7893489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:12.111770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 743/1470] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:12.159800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c273e6c-1bbd-469b-a2e5-cab772af8ddb None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:12.160085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c273e6c-1bbd-469b-a2e5-cab772af8ddb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:12.160354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c273e6c-1bbd-469b-a2e5-cab772af8ddb None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:12.161718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 728/1471] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jun 30 17:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:12.166652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da1730-c0da-499b-b5fb-8fc654806e87 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:12.166943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da1730-c0da-499b-b5fb-8fc654806e87 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:12.167223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da1730-c0da-499b-b5fb-8fc654806e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:12.184727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-77da1730-c0da-499b-b5fb-8fc654806e87 None None] MFA Rules not processed for user `2861607f39d44172851da17c72704358`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:12.185915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-77da1730-c0da-499b-b5fb-8fc654806e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:12.192234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 744/1472] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jun 30 17:47:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:12.199606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c273e6c-1bbd-469b-a2e5-cab772af8ddb None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:12.211108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c273e6c-1bbd-469b-a2e5-cab772af8ddb None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:12.214567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:12.215177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:12.215455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:12.215725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:12.224929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:12.226323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b655ae-231e-4dc2-b751-8ded3767d37f service glance] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:12.235484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 729/1473] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:47:14.352513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287a067-7892-426a-a997-4331efab2a91 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:14.353047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287a067-7892-426a-a997-4331efab2a91 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:14.353047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287a067-7892-426a-a997-4331efab2a91 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:14.354493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 745/1474] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:14.360298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818038af-b264-42b5-8eba-4eb501509b05 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:14.360716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818038af-b264-42b5-8eba-4eb501509b05 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:14.360845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818038af-b264-42b5-8eba-4eb501509b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:14.379173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-818038af-b264-42b5-8eba-4eb501509b05 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:14.380356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-818038af-b264-42b5-8eba-4eb501509b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:14.388043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 730/1475] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:14.411492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287a067-7892-426a-a997-4331efab2a91 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:14.424006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287a067-7892-426a-a997-4331efab2a91 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:14.428451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:14.429204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:14.429487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:14.429759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:14.430510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:14.432238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:47:14.432238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:47:14.432238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:47:14.433373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05015b1f-8307-42fa-9686-fb01252e743e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:14.453274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 746/1476] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jun 30 17:47:14 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:47:16.485783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617e90-f00a-4fbe-83eb-b3a5fc1241e4 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:16.486374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617e90-f00a-4fbe-83eb-b3a5fc1241e4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:16.486374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617e90-f00a-4fbe-83eb-b3a5fc1241e4 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:16.487754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 731/1477] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:16.492938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be869a-c067-4345-88de-112b0249a961 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:16.493368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be869a-c067-4345-88de-112b0249a961 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:16.493510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be869a-c067-4345-88de-112b0249a961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:16.511537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-27be869a-c067-4345-88de-112b0249a961 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:16.512843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-27be869a-c067-4345-88de-112b0249a961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:16.521093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 747/1478] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:16.565690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617e90-f00a-4fbe-83eb-b3a5fc1241e4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:16.579740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617e90-f00a-4fbe-83eb-b3a5fc1241e4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:16.584918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:16.585552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:16.585827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:16.586097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:16.594021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:16.595668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0677b3-26c4-40c5-94eb-3c0316341eb5 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:16.604698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 732/1479] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:47:18.792753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836538-bc11-4ce6-9d07-0f7c1f9f6852 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:18.792753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836538-bc11-4ce6-9d07-0f7c1f9f6852 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:18.792753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d836538-bc11-4ce6-9d07-0f7c1f9f6852 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:18.792753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 748/1480] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:18.797054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116628-ee1c-4cde-9a8c-0c684e0ce2f9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:18.798106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116628-ee1c-4cde-9a8c-0c684e0ce2f9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:18.798212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43116628-ee1c-4cde-9a8c-0c684e0ce2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:18.815960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-43116628-ee1c-4cde-9a8c-0c684e0ce2f9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:18.817158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-43116628-ee1c-4cde-9a8c-0c684e0ce2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:18.825197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 733/1481] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:18.862074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d836538-bc11-4ce6-9d07-0f7c1f9f6852 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:18.872756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d836538-bc11-4ce6-9d07-0f7c1f9f6852 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:18.876694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:18.877352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:18.877634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:18.877910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:18.886767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:18.888189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf05d0-31ae-4bef-afd8-b5fbb61f056e service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:18.898008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 749/1482] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:47:20.995087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd23cf-e2b4-4e34-a964-d986f7a72d7f None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:20.995609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd23cf-e2b4-4e34-a964-d986f7a72d7f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:20.995756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd23cf-e2b4-4e34-a964-d986f7a72d7f None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:20.997015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 734/1483] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:21.002140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e382-d2c5-4ba7-abd7-ad17dd0b2628 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:21.002588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e382-d2c5-4ba7-abd7-ad17dd0b2628 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:21.002736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e382-d2c5-4ba7-abd7-ad17dd0b2628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:21.020107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-edc1e382-d2c5-4ba7-abd7-ad17dd0b2628 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:21.021307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-edc1e382-d2c5-4ba7-abd7-ad17dd0b2628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:21.029141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 750/1484] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:21.067339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd23cf-e2b4-4e34-a964-d986f7a72d7f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:21.078196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd23cf-e2b4-4e34-a964-d986f7a72d7f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:21.082241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:21.082907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:21.083185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:21.083459 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:21.092308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:21.094073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c6059-aad4-401d-9a4c-eed31042cc7c service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:21.103966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 735/1485] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:47:23.227611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c663f-5bb5-4567-9f52-e84516564185 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:23.228207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c663f-5bb5-4567-9f52-e84516564185 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:23.228207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c663f-5bb5-4567-9f52-e84516564185 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:23.229546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 751/1486] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:23.235745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41571ad0-9de1-4a74-8060-412d12fa54e9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:23.236223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41571ad0-9de1-4a74-8060-412d12fa54e9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:23.236334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41571ad0-9de1-4a74-8060-412d12fa54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:23.254456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-41571ad0-9de1-4a74-8060-412d12fa54e9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:23.255601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-41571ad0-9de1-4a74-8060-412d12fa54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:23.263701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 736/1487] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:23.303785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c663f-5bb5-4567-9f52-e84516564185 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:23.316187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c663f-5bb5-4567-9f52-e84516564185 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:23.319693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:23.320578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:23.320862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:23.321140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:23.330118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:23.331625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7f10-a68d-4694-9d3b-4422e5b4c4ce service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:23.340823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 752/1488] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:47:25.523070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe4a-202a-4c2c-a791-4e72d1a0b6cb None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:25.523645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe4a-202a-4c2c-a791-4e72d1a0b6cb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:25.523645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe4a-202a-4c2c-a791-4e72d1a0b6cb None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:25.524970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 737/1489] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jun 30 17:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:25.530115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c8a3-08e0-48ef-b9a3-b9c8181af095 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:25.530570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c8a3-08e0-48ef-b9a3-b9c8181af095 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:25.530712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c8a3-08e0-48ef-b9a3-b9c8181af095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:25.549457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d0e5c8a3-08e0-48ef-b9a3-b9c8181af095 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:25.550688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d0e5c8a3-08e0-48ef-b9a3-b9c8181af095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:25.558610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 753/1490] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jun 30 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:47:25.614267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79181c32-8a65-4a50-bd5d-4aa2e33b1b1d None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:25.614595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79181c32-8a65-4a50-bd5d-4aa2e33b1b1d None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:25.614875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79181c32-8a65-4a50-bd5d-4aa2e33b1b1d None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:25.616281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 738/1491] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jun 30 17:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:47:25.622111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadca77-cf95-4d61-ac5d-0e18e34575b8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:25.622428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadca77-cf95-4d61-ac5d-0e18e34575b8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:25.622709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadca77-cf95-4d61-ac5d-0e18e34575b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:25.658991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-5cadca77-cf95-4d61-ac5d-0e18e34575b8 None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:47:25.660178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5cadca77-cf95-4d61-ac5d-0e18e34575b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:47:25.666451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 754/1492] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jun 30 17:47:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:47:25.675381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79181c32-8a65-4a50-bd5d-4aa2e33b1b1d None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:47:25.685847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79181c32-8a65-4a50-bd5d-4aa2e33b1b1d None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:47:25.689343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:47:25.689969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:47:25.690251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:47:25.690555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:47:25.699614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:47:25.701049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4928a8-46e5-431d-8eaf-7ea7589f3182 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:47:25.711183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 739/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:47:35.655997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=42 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:47:35.706791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=33 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:49:00.650525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.651233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.651233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.669547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:49:00.670783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:00.679102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 755/1494] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:49:00.686955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.687438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.687536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.705300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:49:00.706529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:00.714183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 740/1495] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:49:00.718599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.728661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59014d-fdc9-45bc-bb60-9f2e691aa211 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.733151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.733796 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.734077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.734372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.735092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:49:00.735398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:00.737051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:00.737051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:49:00.737051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:49:00.737938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:00.749171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 756/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:49:00.756985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.765937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d2012-6d96-4cce-a42a-1fffbe877857 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.771152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.771798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.772082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.772357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.773069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:00.774923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:00.774923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:49:00.774923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:49:00.776131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:00.811176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 741/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:49:00 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:49:00.815851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.825968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f1ac-f267-464d-9fb8-c24a3abcb83b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.880299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.880954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.881235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.881510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.882194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:00.883226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:00.883226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:49:00.883226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:49:00.884440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:00.903165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 757/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:49:00 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:49:00.907182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.918014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da46920-1270-432f-97b5-8ce3f1422999 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.922668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.923829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.924111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.924386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.925101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:00.926112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:00.926112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:49:00.926112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:49:00.927024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:00.940165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 742/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:49:00.944112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.954129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dfba9-ac5b-4f4a-8bab-02501aebc9fc admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.958773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.959429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.959717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.959990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:00.960702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:00.961697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:00.961697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:49:00.961697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:49:00.962608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa62d3d-91c2-40bc-8461-941446578393 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:00.975821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 758/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:49:00.980018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:00.991095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872a980-c839-4b49-b753-263164b13406 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:00.995594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:00.996467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:00.996751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:00.997077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] PATH_INFO: `/v3/projects/6636858626d24dc7b83e2f506623baee/users/859b8ed52268488a92d1f256d0025bb6/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.012264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6636858626d24dc7b83e2f506623baee, user_id=859b8ed52268488a92d1f256d0025bb6, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:01.013285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:01.013285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:49:01.013285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:49:01.015266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140185a-98a8-4e7f-8292-1c0dcd9e85c0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:01.028275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 743/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:49:00 2022] PUT /identity/v3/projects/6636858626d24dc7b83e2f506623baee/users/859b8ed52268488a92d1f256d0025bb6/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:49:01.040151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc47765-4b06-4385-8c30-50d8eb74cd92 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.040448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc47765-4b06-4385-8c30-50d8eb74cd92 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.040734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc47765-4b06-4385-8c30-50d8eb74cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.067974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2bc47765-4b06-4385-8c30-50d8eb74cd92 None None] MFA Rules not processed for user `859b8ed52268488a92d1f256d0025bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:49:01.069198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2bc47765-4b06-4385-8c30-50d8eb74cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:01.135367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 759/1502] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:49:01.148347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5bda0-15e3-492c-b089-05100b38632b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.148637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5bda0-15e3-492c-b089-05100b38632b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.148911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5bda0-15e3-492c-b089-05100b38632b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.150428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 744/1503] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jun 30 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:49:01.155371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269138a-f491-42cc-b095-288de2c35505 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.155609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269138a-f491-42cc-b095-288de2c35505 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.155886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269138a-f491-42cc-b095-288de2c35505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.174591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7269138a-f491-42cc-b095-288de2c35505 None None] MFA Rules not processed for user `cd5dd586ede14a918301fd93b4c8f320`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:49:01.175838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7269138a-f491-42cc-b095-288de2c35505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:01.218462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:49:01.228385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 760/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jun 30 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:49:01.234862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5bda0-15e3-492c-b089-05100b38632b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:01.245884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5bda0-15e3-492c-b089-05100b38632b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:01.249358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:01.249985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.250266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.250678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.258764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:01.260176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:01.268765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 745/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:49:01.312017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441eeb-3c0a-4489-b031-0043398505aa None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.312301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441eeb-3c0a-4489-b031-0043398505aa None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.312572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25441eeb-3c0a-4489-b031-0043398505aa None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.313854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 761/1506] 10.222.0.16 () {52 vars in 829 bytes} [Thu Jun 30 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:49:01.317324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:01.376733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cf6dc-ed72-432b-8cf8-0f09bf09bb12 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:01.381181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:01.382031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.382313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.382617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] PATH_INFO: `/v3/users/859b8ed52268488a92d1f256d0025bb6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.383786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=859b8ed52268488a92d1f256d0025bb6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:01.384791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:01.384791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:49:01.384791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:49:01.386008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:01.449264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859b8ed52268488a92d1f256d0025bb6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:01.449871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859b8ed52268488a92d1f256d0025bb6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:01.467919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859b8ed52268488a92d1f256d0025bb6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:01.478716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859b8ed52268488a92d1f256d0025bb6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:01.479571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859b8ed52268488a92d1f256d0025bb6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:01.489222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:49:01.499936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 746/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:49:01 2022] DELETE /identity/v3/users/859b8ed52268488a92d1f256d0025bb6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:49:01.504417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.504707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.504986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.522875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:49:01.524071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:01.581465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 762/1508] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:49:01.590879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:01.592275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:01.649254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccf230-dba7-482c-a1b0-1aad3e9be013 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:01.652786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:01.653423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:01.653705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:01.653980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:01.663368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:01.664804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471aaf0d-0e14-419e-b560-4e0c2cb4e40b service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:01.674090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 747/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:49:02.038309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:49:02.039878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:49:02.056681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65c956-39f9-4340-877d-1faa4cf7793d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:49:02.062601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:49:02.063476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:49:02.063761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:49:02.064570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] PATH_INFO: `/v3/projects/6636858626d24dc7b83e2f506623baee` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:49:02.065768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6636858626d24dc7b83e2f506623baee)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:49:02.066898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:49:02.066898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:49:02.066898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:49:02.068148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:49:02.129843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6636858626d24dc7b83e2f506623baee', 'request_id': 'req-e75801be-7fd4-426a-ad30-eda793670ca9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:02.135786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6636858626d24dc7b83e2f506623baee', 'request_id': 'req-e75801be-7fd4-426a-ad30-eda793670ca9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:02.145562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6636858626d24dc7b83e2f506623baee', 'request_id': 'req-e75801be-7fd4-426a-ad30-eda793670ca9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:02.163696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] The token cache is being invalidate because project 6636858626d24dc7b83e2f506623baee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:49:02.163983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:02.164628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-e75801be-7fd4-426a-ad30-eda793670ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:49:02.166281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 763/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:49:02 2022] DELETE /identity/v3/projects/6636858626d24dc7b83e2f506623baee => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:49:11.669930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=29 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:49:12.160718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:50:44.711890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:44.712488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:44.712488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:44.713801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:50:44.718707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02bab-006d-43af-9db2-053ad7a67037 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:44.719115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02bab-006d-43af-9db2-053ad7a67037 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:44.719259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a02bab-006d-43af-9db2-053ad7a67037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:44.737325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-85a02bab-006d-43af-9db2-053ad7a67037 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:44.738550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-85a02bab-006d-43af-9db2-053ad7a67037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:44.800605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:44.962964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:50:44.964601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:45.037308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697b755-ed5b-4d22-84a3-fffa95ffdb92 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:50:45.043088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:50:45.043725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:45.044006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:45.044279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:45.052141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:50:45.054368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692e2d5-1bdb-41f6-b570-3e9a9333d2f3 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:50:45.065489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 749/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:50:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:50:45.956046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e9c-729c-46c4-ad17-8b398602a81f None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:45.956583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e9c-729c-46c4-ad17-8b398602a81f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:45.956583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de879e9c-729c-46c4-ad17-8b398602a81f None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:45.957932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:50:45.962992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abad01-ed55-468d-ab51-32e8a3a610f2 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:45.963405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abad01-ed55-468d-ab51-32e8a3a610f2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:45.963556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abad01-ed55-468d-ab51-32e8a3a610f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:45.985094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-20abad01-ed55-468d-ab51-32e8a3a610f2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:45.986325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-20abad01-ed55-468d-ab51-32e8a3a610f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:45.995326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:46.149860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de879e9c-729c-46c4-ad17-8b398602a81f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:50:46.164705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de879e9c-729c-46c4-ad17-8b398602a81f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:50:46.170047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:50:46.170717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:46.170998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:46.171271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:46.179187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:50:46.180772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771be08-30fc-4ff6-bd88-7f589ded04d9 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:50:46.189944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 766/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:50:46.660834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b359b6-e0a0-4932-9ac0-0f91f1c777fd None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:46.661118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b359b6-e0a0-4932-9ac0-0f91f1c777fd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:46.661392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b359b6-e0a0-4932-9ac0-0f91f1c777fd None None] PATH_INFO: `/v3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:46.662766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 751/1517] 10.222.0.44 () {54 vars in 925 bytes} [Thu Jun 30 17:50:46 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:50:46.670286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01175da-5151-408e-8a96-6f56cabcf1e8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:46.670607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01175da-5151-408e-8a96-6f56cabcf1e8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:46.670886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01175da-5151-408e-8a96-6f56cabcf1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:46.688437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e01175da-5151-408e-8a96-6f56cabcf1e8 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:46.689649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e01175da-5151-408e-8a96-6f56cabcf1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:46.750395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 767/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 30 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:55.700401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9140d-4446-4aaa-9acb-21a4fe2d382b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:55.700911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9140d-4446-4aaa-9acb-21a4fe2d382b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:55.700911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9140d-4446-4aaa-9acb-21a4fe2d382b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:55.702267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 752/1519] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jun 30 17:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:50:55.708936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83ff4-9c3c-4637-801b-ff1aacaa7f2b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:55.709362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83ff4-9c3c-4637-801b-ff1aacaa7f2b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:55.709491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea83ff4-9c3c-4637-801b-ff1aacaa7f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:55.733112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4ea83ff4-9c3c-4637-801b-ff1aacaa7f2b None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:55.734341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4ea83ff4-9c3c-4637-801b-ff1aacaa7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:55.742727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 768/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jun 30 17:50:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:55.752017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9140d-4446-4aaa-9acb-21a4fe2d382b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:50:55.765116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9140d-4446-4aaa-9acb-21a4fe2d382b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:50:55.769862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:50:55.770521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:55.770801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:55.771075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:55.779636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:50:55.781195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744796eb-2db8-4d87-a3f3-04dff0e078b6 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:50:55.791115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 753/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:50:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:50:57.423528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0cfa9-9d48-4bd0-b0d8-f32a7b0b6c67 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:57.424095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0cfa9-9d48-4bd0-b0d8-f32a7b0b6c67 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:57.424095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0cfa9-9d48-4bd0-b0d8-f32a7b0b6c67 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:57.425462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:50:57.430264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792adaa5-65f2-4dec-b9c2-990fafaac76f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:57.430711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792adaa5-65f2-4dec-b9c2-990fafaac76f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:57.430835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792adaa5-65f2-4dec-b9c2-990fafaac76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:57.448350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-792adaa5-65f2-4dec-b9c2-990fafaac76f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:57.449813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-792adaa5-65f2-4dec-b9c2-990fafaac76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:57.457503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:57.611886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0cfa9-9d48-4bd0-b0d8-f32a7b0b6c67 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:50:57.626173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0cfa9-9d48-4bd0-b0d8-f32a7b0b6c67 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:50:57.631760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:50:57.632398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:57.633213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:57.633486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:57.641555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:50:57.643172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee6b89-ca80-4b99-a6bf-48377dfb0a9c service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:50:57.653063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 770/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:50:58.467232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902787-8187-4e17-bade-252ace17439a None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:58.467805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902787-8187-4e17-bade-252ace17439a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:58.467805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902787-8187-4e17-bade-252ace17439a None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:58.469159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 17:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:50:58.473898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34cbf0-cc11-4d57-a3a6-c0c50532e57c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:58.474321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34cbf0-cc11-4d57-a3a6-c0c50532e57c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:58.474467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34cbf0-cc11-4d57-a3a6-c0c50532e57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:58.493271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9c34cbf0-cc11-4d57-a3a6-c0c50532e57c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:50:58.494510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9c34cbf0-cc11-4d57-a3a6-c0c50532e57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:50:58.502138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 17:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:50:58.657432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17902787-8187-4e17-bade-252ace17439a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:50:58.672866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17902787-8187-4e17-bade-252ace17439a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:50:58.678602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:50:58.679244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:50:58.680113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:50:58.680390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:50:58.689018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:50:58.690639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbfd90-6163-4449-b1c9-71ebd10cb223 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:50:58.700629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 756/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:51:02.691636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:02.692216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:02.692216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:02.693562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:51:02.699845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8a74c-f4a9-4695-99e1-8424fd70775e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:02.700262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8a74c-f4a9-4695-99e1-8424fd70775e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:02.700403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8a74c-f4a9-4695-99e1-8424fd70775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:02.717897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d1a8a74c-f4a9-4695-99e1-8424fd70775e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:51:02.719117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d1a8a74c-f4a9-4695-99e1-8424fd70775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:02.727228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:51:02.764679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:51:02.766370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:02.821584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7fba-24df-44fe-9d98-14504e036b41 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:51:02.826535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:51:02.827579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:02.827870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:02.828150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:02.839939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:51:02.841367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1002fc-afa3-4ddd-973f-ae3b87c6c7fe service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:51:02.846253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:51:02.851854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 773/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:51:04.866725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:04.867296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:04.867296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:04.868655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:51:04.873908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab205-c6f9-4727-be3d-b58a8f299f51 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:04.874353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab205-c6f9-4727-be3d-b58a8f299f51 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:04.874549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab205-c6f9-4727-be3d-b58a8f299f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:04.892867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f0dab205-c6f9-4727-be3d-b58a8f299f51 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:51:04.894080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f0dab205-c6f9-4727-be3d-b58a8f299f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:04.902361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:51:04.939556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:51:04.941238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:04.954885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fb87d-29b0-4b42-bfa3-f4a4b43fbf67 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:51:04.958592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:51:04.959294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:04.959583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:04.959886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:04.969653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:51:04.971100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9873e6d-0d50-464a-ad37-d839c74fdcb4 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:51:04.981027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 759/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:51:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:51:08.525421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:08.525980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:08.525980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:08.527312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:51:08.532476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ba70-5685-403c-9b71-6be55d9dd6cb None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:08.532895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ba70-5685-403c-9b71-6be55d9dd6cb None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:08.533032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ba70-5685-403c-9b71-6be55d9dd6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:08.550790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d772ba70-5685-403c-9b71-6be55d9dd6cb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:51:08.552003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d772ba70-5685-403c-9b71-6be55d9dd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:08.560127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:51:08.859736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:51:08.861416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:08.918589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709581e8-774c-463f-acf6-4cc315c721e5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:51:08.922176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:51:08.922840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:08.923127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:08.923402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:08.932835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:51:08.934263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aba3c-841c-4ea1-b4c9-d7eee57da467 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:51:08.943589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 776/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:51:10.997595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:10.998055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:10.998186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:10.999603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 17:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:51:11.004708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791294-f622-45d6-a4a3-9dcad42c7c25 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:11.005152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791294-f622-45d6-a4a3-9dcad42c7c25 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:11.005261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791294-f622-45d6-a4a3-9dcad42c7c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:11.024008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f2791294-f622-45d6-a4a3-9dcad42c7c25 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:51:11.025201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f2791294-f622-45d6-a4a3-9dcad42c7c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:11.032880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:51:11.334921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:51:11.337233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:51:11.349988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8b4ad-5905-4550-952e-74130730431a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:51:11.353476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:51:11.354100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:51:11.354409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:51:11.354682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:51:11.363282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:51:11.364765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcbc81-9cfb-4fd6-a742-80b6758199b5 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:51:11.374048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 762/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:51:21.018214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=32 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:51:21.369797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=70 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:41.040426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808088cb-9f9f-49c2-af4f-91fe917cf577 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.041067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808088cb-9f9f-49c2-af4f-91fe917cf577 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.041067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808088cb-9f9f-49c2-af4f-91fe917cf577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.064750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-808088cb-9f9f-49c2-af4f-91fe917cf577 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.066169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-808088cb-9f9f-49c2-af4f-91fe917cf577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.079629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 778/1540] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.089050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e26d-859c-4b30-8ab5-0f6fb53659ea None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.089610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e26d-859c-4b30-8ab5-0f6fb53659ea None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.089610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036e26d-859c-4b30-8ab5-0f6fb53659ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.115458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e036e26d-859c-4b30-8ab5-0f6fb53659ea None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.116825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e036e26d-859c-4b30-8ab5-0f6fb53659ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.123660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.124092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.124382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.131277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 763/1541] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.132629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4cc0f-d04a-43a9-9191-44599513a69e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.132913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4cc0f-d04a-43a9-9191-44599513a69e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.133188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4cc0f-d04a-43a9-9191-44599513a69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.156399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.157627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.159604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-56b4cc0f-d04a-43a9-9191-44599513a69e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.160875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-56b4cc0f-d04a-43a9-9191-44599513a69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.175409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 779/1542] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:41.176439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.187949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 764/1543] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.189619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd34b-08c5-4796-adde-c6e606722bf0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.189912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd34b-08c5-4796-adde-c6e606722bf0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.190187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ecd34b-08c5-4796-adde-c6e606722bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.196926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd9dc1-98a5-471f-8bdf-6f0b1143bec1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.206248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.207114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.207960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.208241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.208973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.209287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.210941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a4d6-4d1f-4f29-979b-b14dad1e88c3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.219190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e2ecd34b-08c5-4796-adde-c6e606722bf0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.220559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e2ecd34b-08c5-4796-adde-c6e606722bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.230912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 780/1544] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.232224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.232515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.232798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.242038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 765/1545] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.243351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.243638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.243914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.262262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.267029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.277794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.280404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.288151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 781/1546] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.289729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.294122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 766/1547] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.294974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.300639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f1d93-520a-4bff-8ac5-e34ce05e40f8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.305549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03bb5-471f-4062-8b5e-e604de24c3f4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.305715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.306296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.306607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.306877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.307705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.308844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:41.308844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:52:41.308844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:52:41.310022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafbecc-5465-4f15-9738-e21217159441 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.311204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.311856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.312141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.312419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.313130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.313442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.314476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:41.314476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:52:41.314476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:52:41.315373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31eb6-b26e-4e5a-98fc-0d13ca18c75b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.329859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 767/1548] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.329859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.330172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.331687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.345126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 782/1549] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.346747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b3cf-809a-4ea1-9e07-9330cbebf4e9 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.347273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b3cf-809a-4ea1-9e07-9330cbebf4e9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.347789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2b3cf-809a-4ea1-9e07-9330cbebf4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.349838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.351089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.365789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a2f2b3cf-809a-4ea1-9e07-9330cbebf4e9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.367006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a2f2b3cf-809a-4ea1-9e07-9330cbebf4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.412242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 768/1550] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.412898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.421735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a24349-f85e-4a3b-bff2-a9a6e4331788 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.427719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.428374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.428656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.429409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.429626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.429936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.431590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30e5b-2f48-4193-9b6b-62432e6f5878 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.435835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 783/1551] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.437192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7f5a-16c6-4965-9e92-959be80b4e53 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.437483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7f5a-16c6-4965-9e92-959be80b4e53 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.437820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d7f5a-16c6-4965-9e92-959be80b4e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.448732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 769/1552] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.450609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.451113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.451383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.460506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ec2d7f5a-16c6-4965-9e92-959be80b4e53 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.461717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ec2d7f5a-16c6-4965-9e92-959be80b4e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.468555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.469817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 784/1553] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.470071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.471170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.471463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.471739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.481269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 770/1554] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.482001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.492479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a9dae-7715-43f2-aa17-bd097ed705f1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.493019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.494230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.497021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.497659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.497938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.498206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.499101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.501099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef41719-1ff9-4f22-884d-2ba2f2a46515 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.502412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 785/1555] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.503008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.513549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504f9f5-3fc2-47f8-98a6-35e535b8a7c8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.519227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.519900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.520180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.520452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.521133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.523144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d097c3-3f90-42ac-8086-c799683aa005 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.534860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 771/1556] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.536308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.536609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.536908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.543530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 786/1557] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.544817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf93de-96ed-4738-b5fd-022f19b4a626 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.545103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf93de-96ed-4738-b5fd-022f19b4a626 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.545375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf93de-96ed-4738-b5fd-022f19b4a626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.559113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.560525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.566689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a7cf93de-96ed-4738-b5fd-022f19b4a626 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.568567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a7cf93de-96ed-4738-b5fd-022f19b4a626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.620443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 772/1558] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.621232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.627845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 787/1559] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.630982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49218f7-a571-4993-b3d4-d4e78a42184b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.632140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.632438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.632714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.635913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.636640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.636920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.637222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.637900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.639833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd40f6a-41c3-4e3e-a1a9-5e0d77835a9b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.650353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.651562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.659688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:41.664154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 788/1560] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.664154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.671263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab4b9c-71a1-47dc-b1e0-345573865f2d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.675025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 773/1561] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.675025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.675025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.675025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.676654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:41.694905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:41.696140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:41.729643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.730303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.730608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.730897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.731585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.731887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.733471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.745719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 789/1562] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.746458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.757123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 774/1563] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:41.758035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.758654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f759d-64ef-41d9-a421-443c62577c4f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.764035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.764704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.765000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.765273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.765958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.767830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67881120-de37-4c9e-9a36-a688f9351aab None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.767962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.772309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.772959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.773249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.773523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.774424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.776027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.789628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 775/1564] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:41.789628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.790154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 790/1565] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.791011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.797118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5928-bdaa-4a86-b87e-1678d30dd27d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.800458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ce70e-ba05-4c47-b0f9-7d032418de04 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.802239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.802909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.803190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.803469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.804185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.804494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.805684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.806091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.806317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.806625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.806926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.807638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.807940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.809504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.818689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 776/1566] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.819639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.835120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 791/1567] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.835120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.835386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666a5ec-70d1-4bb9-9947-afd838510e99 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.835386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94321921-3505-4d62-a253-315346eff897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.835386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321921-3505-4d62-a253-315346eff897 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.835386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321921-3505-4d62-a253-315346eff897 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.835766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321921-3505-4d62-a253-315346eff897 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.836355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94321921-3505-4d62-a253-315346eff897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.836686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94321921-3505-4d62-a253-315346eff897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.838603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94321921-3505-4d62-a253-315346eff897 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.840916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a92b3-7123-4b76-b5c6-22e74fa32d8b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.845507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.846160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.846519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.846801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.847509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.849954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.854598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 777/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.854598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94321921-3505-4d62-a253-315346eff897 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.864587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94321921-3505-4d62-a253-315346eff897 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.870137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.870844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.871112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.871389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.872092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.873986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.877277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 792/1569] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.877277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.884297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270f9af-a922-48d4-8fd9-1e1654c9983c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.889006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.889663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.889948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.890227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.891150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:41.891463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.893344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.900094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 778/1570] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:41.900914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.913419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 793/1571] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:41.913419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.922869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540fa4e-fea6-43af-9e77-068c50d5ab93 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.962118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a365-7865-4e14-ad04-8ecdbecc9f44 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.966670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.967329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.967617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.967901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.968611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.970197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.980645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:41.980837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 779/1572] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:41.981396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:41.981652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:41.981819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:41.982041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:41.982795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:41.984744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:41.994748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6626e3-3a6e-4937-99de-f1d93b36c301 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:41.998931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 794/1573] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:41.998931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.002487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.003143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.003429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.003713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.004421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.006323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.006608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb58e4-710b-4ffb-b45a-e220d34c0c64 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.011120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.011740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.012020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.012293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.012949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.014966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.032240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 780/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.033344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.045510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 795/1575] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:41 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.045510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.051828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.065230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.104475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b68d-11bc-477a-84d8-3321489afa1d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.109106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.109909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.110211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.110515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.111065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1509a-e89f-419d-adbc-469eb7b8a4b0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.111189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.113207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.116476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.117128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.117433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.117956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.118486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.120074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.130663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 796/1576] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.131465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.144296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85247639-a9f4-4d73-b866-9c2f1f17aafd admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.144478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 781/1577] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.144478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.200010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.200671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.200952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.201225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.201902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.203975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.206561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97896161-0749-45b9-bb30-d9462dd8eab5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.211619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.212330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.212612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.213014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.213647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.215646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.225131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 797/1578] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.225979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.236506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14de6b-52d8-485b-ba60-1a4fde3acc85 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.241259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.242130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.242459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.242736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] PATH_INFO: `/v3/projects/2ed15171dff7466fab5d482c7fbb7b10/users/63422b2e34eb479f972a181fb37f3608/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.243503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 782/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.247397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.257897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed15171dff7466fab5d482c7fbb7b10, user_id=63422b2e34eb479f972a181fb37f3608, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.259017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:42.259017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:52:42.259017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:52:42.260866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.274295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 798/1580] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:42 2022] PUT /identity/v3/projects/2ed15171dff7466fab5d482c7fbb7b10/users/63422b2e34eb479f972a181fb37f3608/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:42.275242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.308448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f2fc6-46fe-496c-a8c0-7739e6fd23f0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.336148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ef3-ed43-4e8b-94d9-419bac5d2ae4 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.337550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.341354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.342048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.342548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.342689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.343414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.345405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.365096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 799/1581] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.365990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.368668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.369333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.369618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.369893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.370648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.372367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.377293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e8592-8a5c-4714-be8b-2944408ac566 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.381936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.382615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.382902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.383175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.386584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 783/1582] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.386729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.386834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.386925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.392549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.393824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727100b-030c-47f8-ac64-d8010a35eea6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.400322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.401127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.401464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.401784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.402532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.404108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.407450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 800/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/users => generated 302 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.408359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.415131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 784/1584] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.415926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.418705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6650001-9cce-42cd-ac83-f6e457597da2 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.423353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.424009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.424298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.424562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.425237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.425333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50a9ca-8c7f-47da-b545-5241b091b7fc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.427186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1603444-5d9c-4d8e-af52-0db819b6f649 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.429715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.430362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.430643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.430915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.431609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.433257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.444212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 785/1585] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.444770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.449674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 801/1586] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.451295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.451788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.452079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.454965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381e021-2803-493f-b568-b9654a2fdd4f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.460249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.460897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.461180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.461453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.462125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.463168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:42.463168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:52:42.463168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:52:42.464518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.470213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:42.471432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:42.480488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 802/1587] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:42.484009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.485175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 786/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:42.488664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.491960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25098992-2755-402e-a0e8-34b4430dc0a4 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.496618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.497268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.497547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.497820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.497930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c96ad-1c42-4d8f-b1ae-9f06805360e4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.498564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.500135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.502445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.503303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.503580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.503850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] PATH_INFO: `/v3/projects/7c891a43f34e484fa45c50ad9ed5db84/users/559bfe19414945eab1e907cf8f917b34/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.511006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 803/1589] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.511870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.520644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c891a43f34e484fa45c50ad9ed5db84, user_id=559bfe19414945eab1e907cf8f917b34, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.522035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd186-0510-4cf9-8c86-4db4b2ddce33 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.523334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.526985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.527625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.527950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.528322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.529112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.530829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.536978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 787/1590] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:42 2022] PUT /identity/v3/projects/7c891a43f34e484fa45c50ad9ed5db84/users/559bfe19414945eab1e907cf8f917b34/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:42.537715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.544340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 804/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.547651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.560216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987157-18c8-4f49-8ac3-c27ff87b2684 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.600481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec89a0c-ba62-4444-bbb8-99a90ae64dc0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.604959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.605838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.606127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.606513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] PATH_INFO: `/v3/projects/4bedd67c63bc4956bb77fdea8aaeea7a/users/6586f910a79c4e77a1c6f39d019f73ff/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.616042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.616703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.616983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.617265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.618052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.619680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.622096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bedd67c63bc4956bb77fdea8aaeea7a, user_id=6586f910a79c4e77a1c6f39d019f73ff, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.624794 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.630018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 805/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.631129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.640311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 788/1593] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:42 2022] PUT /identity/v3/projects/4bedd67c63bc4956bb77fdea8aaeea7a/users/6586f910a79c4e77a1c6f39d019f73ff/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:42.641114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.646893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335cdec-ef0c-445c-8dfd-f160ce8db0d2 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.689481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.708452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f9394-c69b-4150-8668-f3c3af1a81f8 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.712895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.713139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.713588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.713802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.713912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.714087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.714185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.714392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.714856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.715117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.716445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.716718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.729522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 806/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.729706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 789/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.729811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.729914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.737986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd6bb-7221-4bfc-8e8d-54e5c4951ed9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.743200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.743845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.744131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.744411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.745126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.746755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.758085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 790/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.758919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.769010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca84bc-ca7e-446e-bfe6-b43411743021 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.769180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:42.774811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.775586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.775881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.776157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.776880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.778558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d77372-88d6-4fce-9ee2-8737dfd250b2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.787792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632a927-0f91-4b37-9d46-03dfd24f941f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.789096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 791/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:42.790292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.790608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.790884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.791765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.792641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.792927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.793200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.802273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.803757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.813019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 807/1598] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:42.813769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.816243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] MFA Rules not processed for user `559bfe19414945eab1e907cf8f917b34`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:42.818018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:42.823371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ae5-87c8-4793-b351-b26f68292505 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.828415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.829290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.829572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.829840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] PATH_INFO: `/v3/projects/73169e09f9ea4f92aec0a87cb01625b3/users/f8f558b28f094cb8b833666b44a242f5/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.850903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=73169e09f9ea4f92aec0a87cb01625b3, user_id=f8f558b28f094cb8b833666b44a242f5, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.853629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6990e-fc07-4d11-96b2-f10cd102e9bf admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:42.869834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 808/1599] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:42 2022] PUT /identity/v3/projects/73169e09f9ea4f92aec0a87cb01625b3/users/f8f558b28f094cb8b833666b44a242f5/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:42.870983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.871280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.871560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.904101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] MFA Rules not processed for user `6586f910a79c4e77a1c6f39d019f73ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:42.905524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:42.914620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 792/1600] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:42.914620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:42.922249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322423f-5263-458d-a5b4-ef1a7c52e9ce None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:42.992858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:42.993504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:42.994095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:42.994418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:42.995712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:42.997507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.005835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 809/1601] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.007488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.016879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 793/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:43.016879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.029303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039f59-7e1d-49af-bde3-312f3bddfea7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.029839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706027be-df4c-4e95-ab9a-9501cf7c1911 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.035932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.036223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.037026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.037154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.037321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.037502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.037624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] PATH_INFO: `/v3/projects/28015334d69f46da8516a6297be2cec3/users/973b896d0e4e4bafb67f9508d141881c/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.037913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.038958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.040679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.052791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 810/1603] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:43.053586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.055730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28015334d69f46da8516a6297be2cec3, user_id=973b896d0e4e4bafb67f9508d141881c, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.058798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bee4a0-142c-45d9-af41-49c4cb24857c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.064784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1093f1-5823-41c5-aaef-0ece6e30c5a1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.070949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.072024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.072464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.072711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] PATH_INFO: `/v3/projects/0ccbb24cc2d94bf3be83bc55638fdec1/users/e9ec8fef6a7349aa98bae7e7c3229ec2/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.077486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 794/1604] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:43 2022] PUT /identity/v3/projects/28015334d69f46da8516a6297be2cec3/users/973b896d0e4e4bafb67f9508d141881c/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:43.078810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.079104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.079388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.101809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccbb24cc2d94bf3be83bc55638fdec1, user_id=e9ec8fef6a7349aa98bae7e7c3229ec2, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.105451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b803-7d87-45f7-865f-b3a2c7c16f5b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.108525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] MFA Rules not processed for user `f8f558b28f094cb8b833666b44a242f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.109770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.122423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 811/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:43 2022] PUT /identity/v3/projects/0ccbb24cc2d94bf3be83bc55638fdec1/users/e9ec8fef6a7349aa98bae7e7c3229ec2/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:43.123870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:43.124076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9a328-898f-45b3-ba10-fd4c08326f2d None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.124446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9a328-898f-45b3-ba10-fd4c08326f2d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.124794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9a328-898f-45b3-ba10-fd4c08326f2d None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.126839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 812/1606] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 17:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:43.127710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9a328-898f-45b3-ba10-fd4c08326f2d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.194254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 795/1607] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.198699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.219072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9a328-898f-45b3-ba10-fd4c08326f2d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.226284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.226622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.227054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.227334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.228032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.230245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.247734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:43.265760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 813/1608] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:43.266701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.284890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75adfb-b25c-41c0-b179-b307c30cad19 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.368943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323472a0-0f44-4420-8e3e-786a0b06db78 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.369123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.369930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.370357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.370648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] PATH_INFO: `/v3/projects/eacf1ab0e2244313b8f4f04d5d5202fa/users/2ed5e98a29c44296aeb921af4f521042/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.374091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.375760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.376231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.376842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] PATH_INFO: `/v3/projects/66e3ea2346c743ef96f525d34c031848/users/52a3c04395c64202b10451c270bac9c5/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.387504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacf1ab0e2244313b8f4f04d5d5202fa, user_id=2ed5e98a29c44296aeb921af4f521042, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.390775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb71de-fd8a-49cc-99d8-b794013b4c04 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.397657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e3ea2346c743ef96f525d34c031848, user_id=52a3c04395c64202b10451c270bac9c5, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.400608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dced27-43b6-4425-9846-3c6c4d2e98e6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.405678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 796/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:43 2022] PUT /identity/v3/projects/eacf1ab0e2244313b8f4f04d5d5202fa/users/2ed5e98a29c44296aeb921af4f521042/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:43.407042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e9fd-ef8b-47de-a11f-3e0788af6521 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.407346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e9fd-ef8b-47de-a11f-3e0788af6521 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.407623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e9fd-ef8b-47de-a11f-3e0788af6521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.418864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 814/1610] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:43 2022] PUT /identity/v3/projects/66e3ea2346c743ef96f525d34c031848/users/52a3c04395c64202b10451c270bac9c5/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:43.422488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77b38-b528-44f1-9176-6e2a47dec3e0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.422488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77b38-b528-44f1-9176-6e2a47dec3e0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.422488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77b38-b528-44f1-9176-6e2a47dec3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.437002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f332e9fd-ef8b-47de-a11f-3e0788af6521 None None] MFA Rules not processed for user `973b896d0e4e4bafb67f9508d141881c`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.438381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f332e9fd-ef8b-47de-a11f-3e0788af6521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.441920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-56a77b38-b528-44f1-9176-6e2a47dec3e0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.443203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-56a77b38-b528-44f1-9176-6e2a47dec3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.474906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:43.515206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 815/1611] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.523555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 797/1612] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.523555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2456-4206-4b91-b4db-98bc3b387d7e None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.523761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.523854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2456-4206-4b91-b4db-98bc3b387d7e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.523931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.524013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2456-4206-4b91-b4db-98bc3b387d7e None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.524095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.527249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 798/1613] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jun 30 17:52:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:43.528005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d2456-4206-4b91-b4db-98bc3b387d7e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.542804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d2456-4206-4b91-b4db-98bc3b387d7e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.546799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.547861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.548064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.548529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.548815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.549091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.550316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.552460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96625b64-14f5-454c-b357-8377d146ff79 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.579271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 799/1614] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:43.579271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f5f7c-6778-470c-893e-24ef6a63b3ce None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.579271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f5f7c-6778-470c-893e-24ef6a63b3ce None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.579271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f5f7c-6778-470c-893e-24ef6a63b3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.600756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2d2f5f7c-6778-470c-893e-24ef6a63b3ce None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.601997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2d2f5f7c-6778-470c-893e-24ef6a63b3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.610463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 800/1615] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.611748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae8fe-60c8-4137-9be5-4d345cc49d36 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.612155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae8fe-60c8-4137-9be5-4d345cc49d36 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.612256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae8fe-60c8-4137-9be5-4d345cc49d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.632221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0caae8fe-60c8-4137-9be5-4d345cc49d36 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.633536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0caae8fe-60c8-4137-9be5-4d345cc49d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.641565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 801/1616] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.642873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.643163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.643436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.656419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 816/1617] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.657281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.668931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:43.670564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:43.671560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:43.682297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 802/1618] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jun 30 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:43.683180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.695046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2989c6-d4e9-4719-abb0-09d046d92e52 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.706556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.707811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.708097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.708374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.709105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.710743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.724387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 803/1619] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:43.725111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.759733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f8664-b752-4968-96ef-5535ffd9d870 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.764355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.765307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.765481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.765738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.785953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.787440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.799198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 817/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:43.800062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.809375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5043b-d103-4e12-89f3-ef8b6137e324 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.810289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd1d66-51da-4e6d-9a71-cc8742abe6a0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.813413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.814418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.814750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.814945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.815302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.815613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.815955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.816207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.827338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.828217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.829107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.829983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.841864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 804/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:43.842078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 818/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:43.842744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.842907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.856526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29d8b6-9d95-470e-a259-3790089a9b62 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.858219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855415-b8d3-4e56-b9ad-85995a320f96 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.863529 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.864430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.864836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.865189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.865946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.867785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.868124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.868522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:43.932235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.935000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.947549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 819/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:43.948198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:43.953798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:43.958386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:43.966820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36adc2-11f0-4c76-9034-c7dfce12b828 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:43.970947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 805/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:43.986429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:43.989634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:43.989634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:43.989634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] PATH_INFO: `/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71/users/ea06c37647b54639a328f31b35eee7a4/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.023391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb2dc58a3014be98de5a2a42da6ea71, user_id=ea06c37647b54639a328f31b35eee7a4, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.026186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.050391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 820/1625] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:43 2022] PUT /identity/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71/users/ea06c37647b54639a328f31b35eee7a4/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:44.054765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:44.070537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a674a-8303-411e-be6f-ff00e1ce270a service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:44.120912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:44.210366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:44.215258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.215547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.215823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.216893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.218564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85748f-dde6-4b30-ac52-d4f0ed0743e8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.249240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 806/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:44.253310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:44.269527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e99163-3f7c-485f-b178-e1d25f4393d6 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:44.287528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:44.288433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.288713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.288988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] PATH_INFO: `/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71/users/ea06c37647b54639a328f31b35eee7a4/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.292782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb2dc58a3014be98de5a2a42da6ea71, user_id=ea06c37647b54639a328f31b35eee7a4, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.295729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98af557-6f98-4327-864d-b3bffa44fde8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.309696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430c43-078f-4cfc-a422-009b6394a8a2 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.310039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430c43-078f-4cfc-a422-009b6394a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.310357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430c43-078f-4cfc-a422-009b6394a8a2 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.312113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 807/1627] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 17:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 17:52:44.319187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfca3-f04e-4f55-9fda-58945c74727a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.319478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfca3-f04e-4f55-9fda-58945c74727a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.319832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cfca3-f04e-4f55-9fda-58945c74727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.321334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 821/1628] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:44 2022] PUT /identity/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71/users/ea06c37647b54639a328f31b35eee7a4/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:44.331781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.332728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.333118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.357019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b26cfca3-f04e-4f55-9fda-58945c74727a None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:44.358373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b26cfca3-f04e-4f55-9fda-58945c74727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:44.386189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] MFA Rules not processed for user `ea06c37647b54639a328f31b35eee7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:44.387488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:44.446944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 808/1629] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:44.448307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54eaea-04dd-44b3-ba7c-1754876c54d3 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.448595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54eaea-04dd-44b3-ba7c-1754876c54d3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.449140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54eaea-04dd-44b3-ba7c-1754876c54d3 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.450594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 809/1630] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 17:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:44.457285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.457574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.457849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.474156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:44.509807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:44.511914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:44.515264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 822/1631] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:44.516899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:44.531503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 810/1632] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:44.532849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:44.599173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5708237-2f1b-4929-9d10-b2a8f6062973 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:44.603462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:44.604240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.604626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.604970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.616551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41816-1028-430c-9dcb-1abc71b0b4d4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:44.619939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.621561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.621852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:44.622658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.623479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.623479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.636482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.637847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 823/1633] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:44.638822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f500e-4936-4951-b6cd-48d81c97c989 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.651659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 811/1634] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:44.712691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:44.722022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a274af7-aa6e-48ea-880e-ad96383396aa service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:44.732203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:44.733241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:44.733241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:44.733571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:44.746385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:44.747918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:44.766602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 824/1635] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:45.376454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:45.377061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:45.377061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:45.426514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] MFA Rules not processed for user `e9ec8fef6a7349aa98bae7e7c3229ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:45.431041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:45.529979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:45.583293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 812/1636] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:45.587917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:45.671880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:45.688305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f1b9a-b5cb-4a57-a489-0bf6368fabba None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:45.706280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:45.707661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:45.707962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:45.708233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:45.760182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ececb3-9a83-4b1c-9977-494e167f9dc2 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:45.773652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:45.774533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:45.774822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:45.775237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:45.775937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:45.787275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:45.870387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 825/1637] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:45 2022] POST /identity/v3/projects => generated 405 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:52:45.870387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:45.943779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:45.948042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:45.964534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 813/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:45.965641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:45.997981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a883af-d1b0-43cc-afc6-d7c685518604 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.094692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:46.136647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e110709-b15c-48e1-a613-92ab84d820e7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.150940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.151848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.153251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.153638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.191345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.194418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.195608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.196040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.198638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.202441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.280003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 814/1639] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:45 2022] POST /identity/v3/users => generated 309 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:46.283997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:46.319764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5677b-81ca-463e-84e0-5594a744034c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.335899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.336125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.336493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.336773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.337866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.339146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.340939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.342359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.355497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 826/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:46.374723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 815/1641] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:46.397744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:46.420318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2da6a-b378-48db-8018-ff7b12daff76 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.440187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.441090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.441730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.441730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] PATH_INFO: `/v3/projects/625e9c9acf43406b99df8cbe5cc40cec/users/a982737c402948b4b426376c94b45c93/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.511810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625e9c9acf43406b99df8cbe5cc40cec, user_id=a982737c402948b4b426376c94b45c93, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.517224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.545364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 827/1642] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:46 2022] PUT /identity/v3/projects/625e9c9acf43406b99df8cbe5cc40cec/users/a982737c402948b4b426376c94b45c93/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:46.547480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:46.585768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82006383-0038-4339-a292-93e0a826d9b1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.695377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:46.710640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.711718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.712223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.714422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.715908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.719861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.748701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 816/1643] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:46.753321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:46.771584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:46.784621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d021-8a2e-488b-95cd-c46598122589 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.794177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.795621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.800403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a78a4-4ca1-43b2-a540-3cd7f05de47f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:46.801427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.801971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] PATH_INFO: `/v3/projects/625e9c9acf43406b99df8cbe5cc40cec/users/a982737c402948b4b426376c94b45c93/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.805296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=625e9c9acf43406b99df8cbe5cc40cec, user_id=a982737c402948b4b426376c94b45c93, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.808330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc28c23-2cb2-4ad7-89b9-d35d0b7f5fdd admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.808791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:46.809769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:46.810251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:46.810444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] PATH_INFO: `/v3/users/973b896d0e4e4bafb67f9508d141881c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:46.817627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=973b896d0e4e4bafb67f9508d141881c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:46.819726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:46.859193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 828/1644] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:46 2022] PUT /identity/v3/projects/625e9c9acf43406b99df8cbe5cc40cec/users/a982737c402948b4b426376c94b45c93/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:46.952043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973b896d0e4e4bafb67f9508d141881c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:46.952753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973b896d0e4e4bafb67f9508d141881c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:46.979433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973b896d0e4e4bafb67f9508d141881c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:47.002649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973b896d0e4e4bafb67f9508d141881c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:47.003649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973b896d0e4e4bafb67f9508d141881c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:47.035633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 817/1645] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:46 2022] DELETE /identity/v3/users/973b896d0e4e4bafb67f9508d141881c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:47.040080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:47.040766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:47.041317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:47.089357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:47.090728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:47.229366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 829/1646] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:47.295517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:47.297587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:47.421527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c898-1ee3-480a-bb5a-599b3d76fb89 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:47.428489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:47.429328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:47.429661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:47.429941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:47.441817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:47.443828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:47.478505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 818/1647] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:48.597614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:48.599792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:48.622679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5404b-7ead-47a2-9690-10e5d834ab06 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:48.642555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:48.643545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:48.645239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:48.650513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] PATH_INFO: `/v3/users/559bfe19414945eab1e907cf8f917b34` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:48.655603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559bfe19414945eab1e907cf8f917b34)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:48.658491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: WARNING py.warnings [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:48.658491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: warnings.warn(msg) Jun 30 17:52:48.658491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: Jun 30 17:52:48.660820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:48.701699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:48.703457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:48.732004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a29d7-ab47-4b13-949f-181fa6c7a7e9 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:48.747219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:48.749077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:48.749357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:48.750975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] PATH_INFO: `/v3/projects/28015334d69f46da8516a6297be2cec3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:48.755710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28015334d69f46da8516a6297be2cec3)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:48.760815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: WARNING py.warnings [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 17:52:48.760815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: warnings.warn(msg) Jun 30 17:52:48.760815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: Jun 30 17:52:48.763716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:48.841193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559bfe19414945eab1e907cf8f917b34'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.856579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559bfe19414945eab1e907cf8f917b34'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.857710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559bfe19414945eab1e907cf8f917b34'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.860216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:48.918184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28015334d69f46da8516a6297be2cec3', 'request_id': 'req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.934116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559bfe19414945eab1e907cf8f917b34'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.935228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4024377f-f133-4453-89f3-051ff2434a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559bfe19414945eab1e907cf8f917b34'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.961954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28015334d69f46da8516a6297be2cec3', 'request_id': 'req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:48.971921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:48.976575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28015334d69f46da8516a6297be2cec3', 'request_id': 'req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:49.005830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] The token cache is being invalidate because project 28015334d69f46da8516a6297be2cec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:49.006164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:49.007021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9372f273-2f3b-49b4-8ea0-c7abe890a4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:49.009292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 830/1648] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:48 2022] DELETE /identity/v3/users/559bfe19414945eab1e907cf8f917b34 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:49.010745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 819/1649] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:48 2022] DELETE /identity/v3/projects/28015334d69f46da8516a6297be2cec3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:49.017384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.017727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.017969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267063-2f8b-4d77-bd0d-4ced1a4f4b05 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.018076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.018263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267063-2f8b-4d77-bd0d-4ced1a4f4b05 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.020200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87267063-2f8b-4d77-bd0d-4ced1a4f4b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.070617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:49.071139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-87267063-2f8b-4d77-bd0d-4ced1a4f4b05 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:49.072360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-87267063-2f8b-4d77-bd0d-4ced1a4f4b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:49.073328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:49.205468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 831/1650] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:49.214302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.214828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.215245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.289933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:49.294975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:49.320603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 832/1651] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:49.326172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:49.351984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f962df-cf01-4079-859c-ecb3fe62512e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:49.364008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:49.365388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.365677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.365997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.369445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:49.369777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:49.375565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:49.385368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 820/1652] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:49.413770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 833/1653] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:49.418124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:49.433525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b85a87-5034-4dec-ab4e-63c8adff8425 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:49.485696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:49.486614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.488346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.488732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.492162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:49.506238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:49.535066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:49.539504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:49.596717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 821/1654] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:49.600041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:49.726294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1fded-95dc-4890-99c1-279da6418298 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:49.740867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:49.745217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.745709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.746371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.748361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:49.751175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:49.783480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a284a-ce57-4439-a30e-6fa240c3a96f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:49.795529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 822/1655] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:49.796930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:49.809374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735664-b807-4e0c-9ec8-040b261c217d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:49.817319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:49.818620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.819073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.819492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.820551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:49.822400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2a4dc-e2c8-48c8-8d66-8e07dd9e8c29 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:49.839404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 823/1656] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:49.845143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:49.845521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:49.845725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:49.919469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] MFA Rules not processed for user `a982737c402948b4b426376c94b45c93`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:49.922844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:50.015763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:50.016860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:50.017438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:50.017918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:50.048743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:50.053988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:50.062067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:50.094958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 834/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:50.096073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:50.138894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112639-04ea-4522-b985-ad1a2294803f service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:50.149751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:50.150770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:50.152723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:50.153116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:50.154974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:50.166591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353adb0-e6d3-4dd0-abfe-4bdc5b0c2234 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:50.179914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:50.195541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 824/1658] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:50.204694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 835/1659] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:50.206291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:50.227003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87b0af-ff4e-4cb8-ad1b-035c347d618b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:50.238052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:50.239679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:50.240065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:50.240441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] PATH_INFO: `/v3/projects/49a1c468da8045f4a2c881152b499300/users/d5fcb0f0d0e14d79a8105ece5463e0c9/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:50.285975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a1c468da8045f4a2c881152b499300, user_id=d5fcb0f0d0e14d79a8105ece5463e0c9, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:50.288870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:50.328912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 825/1660] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:50 2022] PUT /identity/v3/projects/49a1c468da8045f4a2c881152b499300/users/d5fcb0f0d0e14d79a8105ece5463e0c9/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:50.334145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:50.334571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:50.334973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:50.365092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] MFA Rules not processed for user `d5fcb0f0d0e14d79a8105ece5463e0c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:50.366528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:50.544288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 836/1661] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:50.545586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:50.908100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:50.908100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:50.952523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b01976-cd25-44df-aefe-ac21688fe0cb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:50.967413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:50.968073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:50.968358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:50.971188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:50.971903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:50.974375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.054797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 826/1662] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:50 2022] POST /identity/v3/projects => generated 363 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:51.060205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:51.144187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128da6c-bd53-4126-9095-23bcef813830 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:51.218680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69b636-1126-4d13-9e3e-f30eb31ca6b1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:51.243957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:51.247683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.248095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.248494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.249599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:51.251689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.293238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:51.294383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.294840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.295261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] PATH_INFO: `/v3/projects/7c891a43f34e484fa45c50ad9ed5db84` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.296286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 827/1663] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:51 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:52:51.299032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c891a43f34e484fa45c50ad9ed5db84)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:51.300443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:51.303677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.347015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29fd3-f9f4-4fe4-9f3a-fc57cb289d10 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:51.357145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:51.358005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.358430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.358817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.359671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:51.361407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.388207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 828/1664] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:51.391605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:51.401376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:51.425118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641840-a8fb-4174-9a77-2aa2401dd220 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:51.431957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c891a43f34e484fa45c50ad9ed5db84', 'request_id': 'req-2c493bed-04b0-48d1-b324-fcdf802c03ef'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:51.443507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c891a43f34e484fa45c50ad9ed5db84', 'request_id': 'req-2c493bed-04b0-48d1-b324-fcdf802c03ef'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:51.456639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c891a43f34e484fa45c50ad9ed5db84', 'request_id': 'req-2c493bed-04b0-48d1-b324-fcdf802c03ef'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:51.460336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:51.481869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] The token cache is being invalidate because project 7c891a43f34e484fa45c50ad9ed5db84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:51.482293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:51.484910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2c493bed-04b0-48d1-b324-fcdf802c03ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:51.486900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 837/1665] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:50 2022] DELETE /identity/v3/projects/7c891a43f34e484fa45c50ad9ed5db84 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:51.491934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2b7d-a950-40f2-83e1-c09115bd2e1f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.492350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2b7d-a950-40f2-83e1-c09115bd2e1f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.493147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2b7d-a950-40f2-83e1-c09115bd2e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.549990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fabe2b7d-a950-40f2-83e1-c09115bd2e1f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:51.551470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fabe2b7d-a950-40f2-83e1-c09115bd2e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:51.640696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:51.642067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.642588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.643085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] PATH_INFO: `/v3/projects/bdc88be1f83a4dab812f5410775eb6b7/users/53815d0575c84582b19adc82183688a0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.699591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc88be1f83a4dab812f5410775eb6b7, user_id=53815d0575c84582b19adc82183688a0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:51.705782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.745341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 829/1666] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:51 2022] PUT /identity/v3/projects/bdc88be1f83a4dab812f5410775eb6b7/users/53815d0575c84582b19adc82183688a0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:51.749110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:51.752647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:51.769953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 838/1667] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:51.776664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.779896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.780998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.893418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:51.903166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:51.937488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df390917-cede-4659-a4ab-4b247ba7e1fd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:51.950885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:51.951755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:51.952149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:51.952532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:51.953374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:51.957647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:51.997206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 830/1668] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:51.998047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.024570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ebbd1-0558-42a8-b79d-ebc0dc6b0565 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.036385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.037318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.037602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.037882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] PATH_INFO: `/v3/projects/bdc88be1f83a4dab812f5410775eb6b7/users/53815d0575c84582b19adc82183688a0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.046898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc88be1f83a4dab812f5410775eb6b7, user_id=53815d0575c84582b19adc82183688a0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.049516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a907-741a-469e-8c28-976ecdc7ef5b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.092343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 831/1669] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:51 2022] PUT /identity/v3/projects/bdc88be1f83a4dab812f5410775eb6b7/users/53815d0575c84582b19adc82183688a0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:52.100391 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.100688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.100970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.125438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 839/1670] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:52.127436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.143830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a202c6-d0b4-4590-bce3-7a5876cd8300 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.179694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] MFA Rules not processed for user `53815d0575c84582b19adc82183688a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:52.180983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:52.377554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.379256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.381528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.384059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.386604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:52.387895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.393119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.393645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 832/1671] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:52.412171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.413966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:52.425296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:52.446582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 840/1672] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:52.448526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.462982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:52.481468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7117c47-bbd3-4e06-9ac7-7561ddefe393 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.490792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.491468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.491757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.492037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.492740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.494742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.538835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc14ca8-d8ad-4b88-88f5-9b7602ae07fa None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.546387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 841/1673] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:52 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:52.548997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.550368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.552057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.552343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.552624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.582126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.583835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.610447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 833/1674] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:52.656901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8be5e-9376-4db6-b806-aad072bc658d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.663068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.663823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.667705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.668003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.668820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.681978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.724495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 842/1675] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:52 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:52:52.728631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.755912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56131-8dcd-4776-a2a9-54c25b77a8b7 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.766081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.767357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.767868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.768248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.769086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.771815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.787347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 834/1676] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:52:52.790964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.805516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4c0a-d851-4763-aebb-5d92b77fb6a6 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.815995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.818824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.819195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.819553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.820551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.822482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfa341-5f38-44be-902c-3c6c0f9d03b6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.857156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 843/1677] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:52:52.862465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:52.883339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa0ec8-577f-4f27-b1b7-09cae1e98a60 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:52.892929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:52.893267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:52.893550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:52.893899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] PATH_INFO: `/v3/projects/a3a247f5863144dba2376d37aea80c08/users/51c4ccac97d248f3af95d3beaddab009/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:52.947550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a247f5863144dba2376d37aea80c08, user_id=51c4ccac97d248f3af95d3beaddab009, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:52.962748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:52.991165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 835/1678] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:52:52 2022] PUT /identity/v3/projects/a3a247f5863144dba2376d37aea80c08/users/51c4ccac97d248f3af95d3beaddab009/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:53.005454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c731ec-e03a-489e-b680-89ffb6957977 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.006850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c731ec-e03a-489e-b680-89ffb6957977 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.006850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c731ec-e03a-489e-b680-89ffb6957977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.051828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-30c731ec-e03a-489e-b680-89ffb6957977 None None] MFA Rules not processed for user `51c4ccac97d248f3af95d3beaddab009`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:53.053834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-30c731ec-e03a-489e-b680-89ffb6957977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:53.184965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 844/1679] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:53.192219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:53.194010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:53.311479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11ea6d-cbdd-4d56-822c-a7e5ca427e80 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:53.324401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:53.325298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.325606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.328943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.350614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:53.352890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91d6e8-b29e-44ad-91a1-b9e3d73f07ab service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:53.380186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 836/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:53.424823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784abfc-7407-4ed0-834a-e17a0cde24d7 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.424823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784abfc-7407-4ed0-834a-e17a0cde24d7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.425101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784abfc-7407-4ed0-834a-e17a0cde24d7 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.427100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 845/1681] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:53.474125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81d88-63d7-4235-a5d3-be53bd87bad6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.474364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81d88-63d7-4235-a5d3-be53bd87bad6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.474580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e81d88-63d7-4235-a5d3-be53bd87bad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.527582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e7e81d88-63d7-4235-a5d3-be53bd87bad6 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:53.532927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e7e81d88-63d7-4235-a5d3-be53bd87bad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:53.707017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 837/1682] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:53.861673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36fbd8-c6ef-483f-a1c9-e1ae67d8f5c7 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.862645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36fbd8-c6ef-483f-a1c9-e1ae67d8f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.863801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36fbd8-c6ef-483f-a1c9-e1ae67d8f5c7 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.865916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 846/1683] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jun 30 17:52:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:53.873211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:53.873572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:53.873857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:53.909783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:53.911579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:53.937575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 838/1684] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jun 30 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:53.949297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36fbd8-c6ef-483f-a1c9-e1ae67d8f5c7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:53.967905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:53.973409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:53.989473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36fbd8-c6ef-483f-a1c9-e1ae67d8f5c7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:54.018484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:54.022165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:54.022492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:54.025712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:54.062458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:54.070778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca7b3f-8de5-49c3-a783-2aa73bcb246b service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:54.116635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 847/1685] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:54.184396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24870b73-0b73-46ca-82c7-479bb6a5a986 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:54.189915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:54.191326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:54.191905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:54.192962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] PATH_INFO: `/v3/users/d5fcb0f0d0e14d79a8105ece5463e0c9` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:54.205421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fcb0f0d0e14d79a8105ece5463e0c9)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:54.208661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:54.316441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fcb0f0d0e14d79a8105ece5463e0c9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:54.317557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fcb0f0d0e14d79a8105ece5463e0c9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:54.354504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fcb0f0d0e14d79a8105ece5463e0c9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:54.372933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fcb0f0d0e14d79a8105ece5463e0c9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:54.373913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fcb0f0d0e14d79a8105ece5463e0c9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:54.395870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 839/1686] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:53 2022] DELETE /identity/v3/users/d5fcb0f0d0e14d79a8105ece5463e0c9 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:54.398032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:54.399138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:54.399643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:54.421383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:54.422902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:54.484401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:54.571127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 848/1687] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:54.628424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:54.632979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:54.785433 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a22ed-3d70-4d5a-a4c4-3e7473376495 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:54.791667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:54.792453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:54.792983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:54.793387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:54.809128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:54.810963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c321291-fb46-484e-8870-4d6b44efab4a service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:54.840831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 840/1688] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:55.174510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:55.176933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:55.196794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b662130-0a93-471e-b028-8514dd648ad9 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:55.204503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:55.205589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:55.205967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:55.206613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] PATH_INFO: `/v3/users/e9ec8fef6a7349aa98bae7e7c3229ec2` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:55.208373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ec8fef6a7349aa98bae7e7c3229ec2)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:55.211015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:55.308078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ec8fef6a7349aa98bae7e7c3229ec2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:55.326364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ec8fef6a7349aa98bae7e7c3229ec2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:55.328036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ec8fef6a7349aa98bae7e7c3229ec2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:55.386382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ec8fef6a7349aa98bae7e7c3229ec2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:55.388434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ec8fef6a7349aa98bae7e7c3229ec2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:55.453329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 849/1689] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:55 2022] DELETE /identity/v3/users/e9ec8fef6a7349aa98bae7e7c3229ec2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:56.253053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:56.254133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:56.254133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:56.315447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] MFA Rules not processed for user `63422b2e34eb479f972a181fb37f3608`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:56.317975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:56.332608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:56.447343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 841/1690] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:56.467665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:56.470253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:56.518610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:56.520180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:56.659650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c245c6-9686-4457-b12f-27e500d49eb0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:56.686523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:56.687476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:56.687802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:56.688064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] PATH_INFO: `/v3/projects/0ccbb24cc2d94bf3be83bc55638fdec1` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:56.689438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccbb24cc2d94bf3be83bc55638fdec1)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:56.693035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:56.694318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cddb-ec02-4cdb-a67a-b13f790b4526 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:56.712820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:56.713861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:56.714366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:56.714926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:56.734200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:56.739933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:56.765904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 850/1691] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:56.767918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:56.768157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:56.790145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccbb24cc2d94bf3be83bc55638fdec1', 'request_id': 'req-667a8be0-7598-42af-be79-13ae0b51e83b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:56.793537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3190c6-0d79-47e4-9fb1-cf256b869258 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:56.799249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccbb24cc2d94bf3be83bc55638fdec1', 'request_id': 'req-667a8be0-7598-42af-be79-13ae0b51e83b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:56.815287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccbb24cc2d94bf3be83bc55638fdec1', 'request_id': 'req-667a8be0-7598-42af-be79-13ae0b51e83b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:56.843692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] The token cache is being invalidate because project 0ccbb24cc2d94bf3be83bc55638fdec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:56.844306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:56.845332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:56.852803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 842/1692] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:56 2022] DELETE /identity/v3/projects/0ccbb24cc2d94bf3be83bc55638fdec1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:56.854192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:56.858685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:56.892316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:56.953159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:56.977057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:56.977946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:56.982070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:56.982536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] PATH_INFO: `/v3/projects/49a1c468da8045f4a2c881152b499300` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:56.985076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a1c468da8045f4a2c881152b499300)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:56.987085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:57.006909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a8be0-7598-42af-be79-13ae0b51e83b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:57.014931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:57.016371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:57.017103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:57.017608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] PATH_INFO: `/v3/users/a982737c402948b4b426376c94b45c93` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:57.019998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a982737c402948b4b426376c94b45c93)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:57.022530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:57.131653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a982737c402948b4b426376c94b45c93'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.133528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a982737c402948b4b426376c94b45c93'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.169656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a982737c402948b4b426376c94b45c93'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.191126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a982737c402948b4b426376c94b45c93'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.191889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a982737c402948b4b426376c94b45c93'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.235538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 843/1693] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:56 2022] DELETE /identity/v3/users/a982737c402948b4b426376c94b45c93 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:57.238533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a1c468da8045f4a2c881152b499300', 'request_id': 'req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.263477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a1c468da8045f4a2c881152b499300', 'request_id': 'req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.298777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a1c468da8045f4a2c881152b499300', 'request_id': 'req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.336977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] The token cache is being invalidate because project 49a1c468da8045f4a2c881152b499300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:57.337212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.337943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-29c9d8c5-a0f1-4cd1-a2e2-fb1346594083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.343078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 851/1694] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:56 2022] DELETE /identity/v3/projects/49a1c468da8045f4a2c881152b499300 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:57.344549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:57.347801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:57.517882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c139e-be11-4078-991d-e8e021d1394c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:57.527348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:57.528265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:57.528551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:57.528829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] PATH_INFO: `/v3/users/53815d0575c84582b19adc82183688a0` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:57.535682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53815d0575c84582b19adc82183688a0)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:57.538233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:57.651794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:57.652319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:57.652413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:57.655436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53815d0575c84582b19adc82183688a0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.657314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53815d0575c84582b19adc82183688a0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.678821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53815d0575c84582b19adc82183688a0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.683888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:57.690217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53815d0575c84582b19adc82183688a0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.692650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f965f731-e1a8-4da2-a91d-e59ca9375697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53815d0575c84582b19adc82183688a0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:57.710512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] MFA Rules not processed for user `52a3c04395c64202b10451c270bac9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:57.711997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:57.726605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 844/1695] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:52:57 2022] DELETE /identity/v3/users/53815d0575c84582b19adc82183688a0 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:57.837320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 852/1696] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:57.838054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:57.838377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:57.838785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:57.902314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] MFA Rules not processed for user `52a3c04395c64202b10451c270bac9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:57.930917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.118553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:58.120222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.129765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 845/1697] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:58.146294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:58.148733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.158155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59bc7a-7b89-46a5-adcb-ca248b078c29 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:58.276524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:58.282608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404d6c-d85d-4548-b5cc-40464dc02a23 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:58.291322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:58.292722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:58.293017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:58.293642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] PATH_INFO: `/v3/projects/625e9c9acf43406b99df8cbe5cc40cec` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:58.296033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625e9c9acf43406b99df8cbe5cc40cec)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:58.297853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:58.297853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:58.298031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:58.298449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:58.300162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:58.348315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:58.356559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea1ebf-da4d-4060-9754-89e3d2faa16a service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:58.408578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 846/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:58.462423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:58.462735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:58.462993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:58.481447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625e9c9acf43406b99df8cbe5cc40cec', 'request_id': 'req-ea29b410-19ef-4e40-98ea-017dfa565754'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:58.496947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625e9c9acf43406b99df8cbe5cc40cec', 'request_id': 'req-ea29b410-19ef-4e40-98ea-017dfa565754'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:58.524682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] MFA Rules not processed for user `2ed5e98a29c44296aeb921af4f521042`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:58.525882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.526076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625e9c9acf43406b99df8cbe5cc40cec', 'request_id': 'req-ea29b410-19ef-4e40-98ea-017dfa565754'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:58.571885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] The token cache is being invalidate because project 625e9c9acf43406b99df8cbe5cc40cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:58.572178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:58.576473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ea29b410-19ef-4e40-98ea-017dfa565754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:58.580785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 853/1699] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:58 2022] DELETE /identity/v3/projects/625e9c9acf43406b99df8cbe5cc40cec => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:52:58.586370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d630-2938-4096-a812-ab2281eb9bb0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:58.586670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d630-2938-4096-a812-ab2281eb9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:58.586932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821d630-2938-4096-a812-ab2281eb9bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:58.660973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 847/1700] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:52:58.692527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:58.693885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.703900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:58.871175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b52ae1-8d11-4684-92bb-98a1221ec0e4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:58.878757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3821d630-2938-4096-a812-ab2281eb9bb0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:58.880004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3821d630-2938-4096-a812-ab2281eb9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:58.883588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:58.884269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:58.884744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:58.885025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:58.899742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:58.901357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:58.946394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 848/1701] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:52:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:52:58.947200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:58.950728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.056600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 854/1702] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:59.069072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.069420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.069633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.118109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:59.119288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.126065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af690a-e271-4220-a429-45b20eb87cb5 service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.145815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:59.146829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.147119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.147414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] PATH_INFO: `/v3/projects/bdc88be1f83a4dab812f5410775eb6b7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.148886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc88be1f83a4dab812f5410775eb6b7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:59.150854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:59.157283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 855/1703] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:59.159025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:59.198987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079b77-1cd3-4c86-bb4d-5f272e3046f5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.214133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:59.214860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.215146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.215420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.216293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:52:59.216603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:59.218249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:59.262488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 856/1704] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:52:59.263693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:59.318249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ed42-613c-45a7-a61d-397cd5d22771 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.325780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc88be1f83a4dab812f5410775eb6b7', 'request_id': 'req-a05a0233-b66b-4955-9a46-760a34dbfd3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:59.350460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc88be1f83a4dab812f5410775eb6b7', 'request_id': 'req-a05a0233-b66b-4955-9a46-760a34dbfd3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:59.363826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc88be1f83a4dab812f5410775eb6b7', 'request_id': 'req-a05a0233-b66b-4955-9a46-760a34dbfd3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:59.395617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] The token cache is being invalidate because project bdc88be1f83a4dab812f5410775eb6b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:52:59.395910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:59.398684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a05a0233-b66b-4955-9a46-760a34dbfd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:52:59.406046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 849/1705] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:52:58 2022] DELETE /identity/v3/projects/bdc88be1f83a4dab812f5410775eb6b7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:52:59.407603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8663d1-bd57-4a79-b1a9-a983e6b51dba None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.408082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8663d1-bd57-4a79-b1a9-a983e6b51dba None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.408678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8663d1-bd57-4a79-b1a9-a983e6b51dba None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.410447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 850/1706] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 17:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:52:59.411946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.412452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.412909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.470625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:59.477316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.518674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:59.577891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:59.581060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.581976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.582553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.583668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:59.586325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964eb8b-3276-4e79-bde2-44d323f2741d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:59.598873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:52:59.633092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 857/1707] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/projects => generated 371 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:59.635859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574eee4-3894-4be7-b50c-077c7120aa81 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.636163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574eee4-3894-4be7-b50c-077c7120aa81 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.637813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574eee4-3894-4be7-b50c-077c7120aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.673740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-1574eee4-3894-4be7-b50c-077c7120aa81 None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:59.678346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1574eee4-3894-4be7-b50c-077c7120aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.698365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 851/1708] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:59.703540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:59.709323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.730233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ea6e7-18c0-4c4f-a434-02327235c24a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.860404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 858/1709] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:59.863598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.863878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.864159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.866354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:59.870443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.870443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.870443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.870443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:59.871914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:52:59.922901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:52:59.929254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:52:59.941610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 852/1710] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:52:59.942973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:59.958697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387b632-157e-4427-be94-003df15507e4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.969723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 859/1711] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:52:59.974370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:52:59.978286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:52:59.979183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:52:59.979619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:52:59.980047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:52:59.980926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:52:59.987269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e6db8-fadd-4f6a-ae64-5163948c58d6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:52:59.988560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.006684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.007533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.007922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.008433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.009285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:00.009723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.011482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.012259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 853/1712] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:00.016592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.040474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 860/1713] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:53:00.041160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31d564-6907-410b-9342-465b743cf489 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.042778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.054205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.055376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.055825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.056247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.057283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.058940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.064731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa84bc5-ef87-4256-9a71-2efecdabada7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.084639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.085386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.085773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.086105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.087238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.095610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.097857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 854/1714] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:00.098902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.125547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a06b6-16db-4f09-a014-027218fd35d2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.133696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.134878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.135302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.135672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] PATH_INFO: `/v3/projects/f770c75aaa8e40098c18a21c8d4511be/users/bc6b7f1dc44548cbbff7121bb6f52801/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.172628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 861/1715] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:00 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:00.177419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.201853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f770c75aaa8e40098c18a21c8d4511be, user_id=bc6b7f1dc44548cbbff7121bb6f52801, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.208362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.255759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 855/1716] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:00 2022] PUT /identity/v3/projects/f770c75aaa8e40098c18a21c8d4511be/users/bc6b7f1dc44548cbbff7121bb6f52801/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:00.256995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.266118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:00.380641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6871d09-2ad6-4ec0-8da8-af9a129692d7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.432795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c49d7-9288-4a6d-8b42-b666170db90a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.441145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.441960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.442365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.442764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.467012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.468540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f5a34-43a7-419c-b592-7e37fe6e3f4e service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.487332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 856/1717] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:00.490892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.494555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.496152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.511250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:00.555910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.556675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.556960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.557249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.557974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.568457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.576717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] MFA Rules not processed for user `bc6b7f1dc44548cbbff7121bb6f52801`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:00.578208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:00.604167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:00.633432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 862/1718] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:00 2022] POST /identity/v3/users => generated 301 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:00.657300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.701330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c678a80-0b5d-400a-b215-a4254eec1fe9 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.715606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.716301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.716591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.716870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.717672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.719524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.742291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 863/1719] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:00.753840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.780912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 857/1720] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:00.784079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.784945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612254bd-1a9c-4a33-bc6d-c9c900e79491 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.799605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.800285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.800573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.801014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.801917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.804277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.818502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999622df-b0c9-478a-92da-79a23c4a5b80 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.830630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.831480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.832576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.832701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.833522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.836358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:00.846103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 864/1721] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:00.847709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.872916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de836b75-3af1-43e9-b7ac-f5c849c75093 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:00.899685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:00.901689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:00.906890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:00.907412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] PATH_INFO: `/v3/projects/88c89834908c41dcbbd2d784d033215d/users/2b46599ea79840b4869d4b1086b3ae25/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:00.928619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 858/1722] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:00 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:00.929768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:00.947469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c89834908c41dcbbd2d784d033215d, user_id=2b46599ea79840b4869d4b1086b3ae25, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:00.950557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0b587-8d90-4f1c-a057-7a4408b0f3d6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.001140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 865/1723] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:00 2022] PUT /identity/v3/projects/88c89834908c41dcbbd2d784d033215d/users/2b46599ea79840b4869d4b1086b3ae25/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:01.003328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.003328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.003990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.070252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] MFA Rules not processed for user `2b46599ea79840b4869d4b1086b3ae25`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:01.071624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:01.074236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dc6e-4298-4f9a-b227-e72503d51670 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.207005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.208269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.208568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.208849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.209555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.211569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.271553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 859/1724] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:00 2022] POST /identity/v3/users => generated 292 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:01.272771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.291818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf3b04-19f2-4e66-9583-10b03c5a018a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.299466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 866/1725] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:01.302125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.304934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.305676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.305790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.306035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.306840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.308445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.317436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:01.330093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5c85c-0448-4edb-a2d2-5cf4bb5bf121 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.337592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 860/1726] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:01.339291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.346877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.349460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.350892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.352820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.354111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.359112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.361237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab6bda-8247-4341-b0be-57e26a7236b9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.372374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.373436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.373700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.373865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] PATH_INFO: `/v3/projects/6420e518e3324af7b2e4f70426b9dc37/users/18acadfb64174598a0da8e4f201704bc/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.412167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6420e518e3324af7b2e4f70426b9dc37, user_id=18acadfb64174598a0da8e4f201704bc, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.415229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.432802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 867/1727] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:01 2022] POST /identity/v3/projects => generated 399 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:01.435920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.457741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 861/1728] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:01 2022] PUT /identity/v3/projects/6420e518e3324af7b2e4f70426b9dc37/users/18acadfb64174598a0da8e4f201704bc/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:01.461417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.626254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf2ef7-81e2-4ec8-b8de-a0d7e47129bc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.659114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.659935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.660224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.660523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.661259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.662902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.687443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134319b-aee2-4b03-8bed-9ed05b114433 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.703821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 862/1729] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:01.705968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.708819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.709528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.709690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.709965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.710700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.712655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.736742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23270d92-468f-409c-8da0-10fcac56a65c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.759505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.760745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.761519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.763550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] PATH_INFO: `/v3/projects/6420e518e3324af7b2e4f70426b9dc37/users/18acadfb64174598a0da8e4f201704bc/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.777645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6420e518e3324af7b2e4f70426b9dc37, user_id=18acadfb64174598a0da8e4f201704bc, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.780484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc92c7-c3ca-4d43-bc34-5366a1cff460 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:01.785607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 868/1730] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:01 2022] POST /identity/v3/users => generated 306 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:01.789778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:01.815428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:01.822805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96815f-1c7d-4138-8531-97d67831216a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:01.831930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 863/1731] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:01 2022] PUT /identity/v3/projects/6420e518e3324af7b2e4f70426b9dc37/users/18acadfb64174598a0da8e4f201704bc/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:01.842261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.842261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.842590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.915018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] MFA Rules not processed for user `18acadfb64174598a0da8e4f201704bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:01.916747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:01.978953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:01.981380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:01.983236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:01.983826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:01.992845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:01.996714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:02.031038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 869/1732] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:02.033605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:02.071331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8470444-96d5-4be3-97e2-f09b0266cb4a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:02.087930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:02.090257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:02.090995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:02.092314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] PATH_INFO: `/v3/projects/4edc9081ae114fce88794185f5f099dd/users/9a34e2293aec48f9a85b3edd60ad3bb0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:02.134752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edc9081ae114fce88794185f5f099dd, user_id=9a34e2293aec48f9a85b3edd60ad3bb0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:02.141420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:02.148388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 864/1733] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:02.161133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:02.163829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:02.177978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 870/1734] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:02 2022] PUT /identity/v3/projects/4edc9081ae114fce88794185f5f099dd/users/9a34e2293aec48f9a85b3edd60ad3bb0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:02.184973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:02.294619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:02.336981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e3e5f-4a32-4c54-8067-71a48e11f932 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:02.369150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd441bc-4a8b-47fe-9aaf-e6848e27d5c9 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:02.382781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:02.383473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:02.385586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:02.387114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:02.388001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:02.391980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:02.429839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 871/1735] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:02.433558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:02.459977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd37b-fd57-43bf-b8fd-e78af7851605 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:02.487486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:02.489356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:02.489787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:02.490228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:02.500295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:02.500568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:02.500857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:02.501120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] PATH_INFO: `/v3/projects/4edc9081ae114fce88794185f5f099dd/users/9a34e2293aec48f9a85b3edd60ad3bb0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:02.506673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edc9081ae114fce88794185f5f099dd, user_id=9a34e2293aec48f9a85b3edd60ad3bb0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:02.510752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e28d9-766e-4868-8248-75aa5e2d8497 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:02.537561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 872/1736] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:02 2022] PUT /identity/v3/projects/4edc9081ae114fce88794185f5f099dd/users/9a34e2293aec48f9a85b3edd60ad3bb0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:02.539152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:02.539449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:02.539735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:02.609974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] MFA Rules not processed for user `9a34e2293aec48f9a85b3edd60ad3bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:02.613858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:02.680709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:02.684683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:02.711795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 865/1737] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:02.717054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:02.719426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:02.763882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:02.843269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 873/1738] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:02.922864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349832ed-6253-4568-95da-16d6911ce220 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:03.010745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:03.012514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.134389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:03.215387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:03.216056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.216337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.218750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.222952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8bba0-5379-47dd-8a7d-213d5a00a4a7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:03.223697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.228842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:03.235346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.235959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.236243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] PATH_INFO: `/v3/users/f8f558b28f094cb8b833666b44a242f5` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.237719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f558b28f094cb8b833666b44a242f5)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:03.239310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:03.239838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:03.241825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:03.266001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 866/1739] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:03.266963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:03.272779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.345068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f558b28f094cb8b833666b44a242f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:03.365278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f558b28f094cb8b833666b44a242f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:03.365945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f558b28f094cb8b833666b44a242f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:03.395311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f558b28f094cb8b833666b44a242f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:03.400980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36189e72-9249-423c-a372-946cf5c4229a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f558b28f094cb8b833666b44a242f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:03.422150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 874/1740] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:03 2022] DELETE /identity/v3/users/f8f558b28f094cb8b833666b44a242f5 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:03.422150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.422150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.422150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.427471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957292d6-cd20-47e6-a9ca-f89669185a66 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:03.465765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] MFA Rules not processed for user `2ed5e98a29c44296aeb921af4f521042`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:03.467121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.578129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:03.578129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.579957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.579957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.583036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.608107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 875/1741] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:03.613332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:03.619500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.648943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a34c8f-a3aa-4f05-b234-fd626e9900f0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:03.663140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:03.663818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.664104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.664476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.672545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.686265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:03.720033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:03.721648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e489f-189a-4adf-9f66-65db235e831d service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:03.755525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 867/1742] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:03.756714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.758667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.758822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.817052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:03.819240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.820282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:03.824737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6ed5b-2497-458e-873e-9cc8e493eea9 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:03.851571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 876/1743] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:03.854966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:03.856692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:03.857048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:03.872892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:03.901717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 868/1744] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:03.912827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:03.916215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:03.921961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] MFA Rules not processed for user `63422b2e34eb479f972a181fb37f3608`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:03.926402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:04.074838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 877/1745] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:04.091420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc24a2f-5a66-452f-9b87-d441077c8a92 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:04.100465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:04.101139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:04.101424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:04.101735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:04.112113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:04.114521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdcc07-8f94-4029-b7c8-fd81ffe5f195 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:04.128826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 869/1746] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:04.361350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:04.365939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:04.402585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29747c9d-d8bc-49c3-bbb1-074e8201a762 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:04.412629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:04.413302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:04.413659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:04.413840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:04.419870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:04.602801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:04.604250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e45b-60f0-45ee-9fbb-ff902d2b2a92 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:04.633256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 878/1747] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:05.367508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:05.368030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:05.368111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:05.369435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 870/1748] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 17:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:53:05.379614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:05.380175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:05.380175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:05.431223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:05.433745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:05.454209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 879/1749] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:53:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:05.469052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:05.471518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:05.508201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3018d7-e8af-4732-968f-73e57cfb1dd9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:05.512260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:05.513099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:05.513544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:05.513972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:05.529018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:05.547573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67763b-57c9-4b34-a74e-b931f018b543 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:05.585239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 871/1750] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:05.910627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:05.912173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:05.937502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9cc4f-de81-472e-b901-5c93b06f12a6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:05.957398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:05.958300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:05.958798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:05.959091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] PATH_INFO: `/v3/projects/73169e09f9ea4f92aec0a87cb01625b3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:05.962258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73169e09f9ea4f92aec0a87cb01625b3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:05.965385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:06.104793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:06.126117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73169e09f9ea4f92aec0a87cb01625b3', 'request_id': 'req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:06.137892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73169e09f9ea4f92aec0a87cb01625b3', 'request_id': 'req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:06.161495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73169e09f9ea4f92aec0a87cb01625b3', 'request_id': 'req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:06.185231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] The token cache is being invalidate because project 73169e09f9ea4f92aec0a87cb01625b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:06.185537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:06.189123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9222c7ae-b2aa-4980-91ee-99ca40ab77e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:06.196223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 880/1751] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:05 2022] DELETE /identity/v3/projects/73169e09f9ea4f92aec0a87cb01625b3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:06.200710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:06.201005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:06.201292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:06.244156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:06.244156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:06.554184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 872/1752] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:06.561698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:06.562135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:06.562427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:06.632046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:06.643507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:06.675449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 881/1753] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:06.683010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:06.716374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba1695-49c1-452e-914e-6b7431c59445 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:06.735050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:06.737853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:06.738353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:06.742449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:06.784644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:06.785861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:06.796671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:06.859860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 873/1754] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:06.860560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:06.898202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586b4ab-7eae-46f9-8d2c-d27bfe8ae0a8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:06.935345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:06.936039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:06.936338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:06.942417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:06.946487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:06.964501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad51844-839e-4e2f-8980-0ac0773cdfcc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:06.991744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:06.994719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.042837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3224-63e8-4291-b61e-eeacdb17cc65 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:07.062634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:07.063531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:07.063812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:07.064082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] PATH_INFO: `/v3/users/6586f910a79c4e77a1c6f39d019f73ff` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:07.076146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6586f910a79c4e77a1c6f39d019f73ff)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:07.078126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:07.111561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 882/1755] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:06 2022] POST /identity/v3/projects => generated 405 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:07.119570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5960-3a29-4d27-8c2c-5df0edb99fc5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:07.123508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5960-3a29-4d27-8c2c-5df0edb99fc5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:07.125686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa5960-3a29-4d27-8c2c-5df0edb99fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:07.212489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a7aa5960-3a29-4d27-8c2c-5df0edb99fc5 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:07.221321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a7aa5960-3a29-4d27-8c2c-5df0edb99fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.283662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6586f910a79c4e77a1c6f39d019f73ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:07.286899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6586f910a79c4e77a1c6f39d019f73ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:07.326556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:07.353363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6586f910a79c4e77a1c6f39d019f73ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:07.375715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6586f910a79c4e77a1c6f39d019f73ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:07.385133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6586f910a79c4e77a1c6f39d019f73ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:07.434991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 874/1756] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:06 2022] DELETE /identity/v3/users/6586f910a79c4e77a1c6f39d019f73ff => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:07.435917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:07.440281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.473085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 883/1757] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:07.476946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:07.477335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:07.478894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:07.556263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:07.557512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.715224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 884/1758] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:07.717211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:07.721350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.739208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd7985-5b88-467f-902f-4e3056b34507 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:07.760241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:07.766136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:07.766441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:07.766868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:07.772188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:07.774629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:07.845123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:07.855913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 875/1759] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:07 2022] POST /identity/v3/users => generated 303 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:07.857005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:07.866656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:07.906532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e019f-d4f2-4781-bd93-3a24ddc78a80 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:07.922678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:07.923358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:07.923690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:07.932951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.007506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617e1d-c5f7-4915-8d3b-74a609ea303e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:08.021789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:08.022724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.023576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.024012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.044098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.051229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.083352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 876/1760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:08.084462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.097083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.100121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.106208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071160f-7d3f-43f5-b3c7-3053c8f284d1 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:08.115508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:08.116283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.117083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.117571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.118468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.125442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.125795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 885/1761] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 17:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 17:53:08.150557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 877/1762] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:08.151463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.167717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a9eed-0761-4090-9416-effb1d549493 service swift] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:08.195118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:08.195812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.196095 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.196368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.200336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.201723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a65bd9-08f7-4c9f-8ecb-0b1ea4709907 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.239621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 886/1763] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:08.244629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.261109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f140136-b94e-416e-aa69-331eadf6b408 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:08.274773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:08.275785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.276072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.276355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] PATH_INFO: `/v3/projects/65e6f580e58644e5a9a95ddc572212e9/users/bc8cd03051a64d17bccb5b5bdaa45898/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.338814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65e6f580e58644e5a9a95ddc572212e9, user_id=bc8cd03051a64d17bccb5b5bdaa45898, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.351507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.402735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 878/1764] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:08 2022] PUT /identity/v3/projects/65e6f580e58644e5a9a95ddc572212e9/users/bc8cd03051a64d17bccb5b5bdaa45898/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:08.414813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.415200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.415599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.531630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] MFA Rules not processed for user `bc8cd03051a64d17bccb5b5bdaa45898`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:08.533352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:08.654437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 887/1765] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:08.659275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.800516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:08.869438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca48fca-4f38-49d4-8c30-5160f9947f86 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:08.876420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:08.877594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:08.878083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:08.878578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:08.879455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:08.881552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:08.946984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 879/1766] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:08 2022] POST /identity/v3/projects => generated 405 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:08.952164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.972315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:08.981219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:08.991926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e34d-4d4c-4178-8aec-ca519f14c349 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.129450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d184c-5c06-468d-b4a8-957d863429b5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.134849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.135832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.137687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.137687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] PATH_INFO: `/v3/projects/4bedd67c63bc4956bb77fdea8aaeea7a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.138075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bedd67c63bc4956bb77fdea8aaeea7a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.140978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.144409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.145285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.145720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.146151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.147033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.149495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.164510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:09.189055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 888/1767] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:08 2022] POST /identity/v3/users => generated 309 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:09.193170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.226658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530077-504b-4a3c-866f-ecc453dff151 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.237014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.238013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.238396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.238782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.241113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.243534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.263826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bedd67c63bc4956bb77fdea8aaeea7a', 'request_id': 'req-1c10e5d5-d73b-48b5-8373-fdcef852a45b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.272584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 889/1768] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:09.274989 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.288082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bedd67c63bc4956bb77fdea8aaeea7a', 'request_id': 'req-1c10e5d5-d73b-48b5-8373-fdcef852a45b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.295854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c2886-0b4c-481c-9993-32850f96cefc admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.300178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bedd67c63bc4956bb77fdea8aaeea7a', 'request_id': 'req-1c10e5d5-d73b-48b5-8373-fdcef852a45b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.321546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] The token cache is being invalidate because project 4bedd67c63bc4956bb77fdea8aaeea7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:09.321888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.322702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.328893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 880/1769] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:08 2022] DELETE /identity/v3/projects/4bedd67c63bc4956bb77fdea8aaeea7a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:09.337936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.340337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.412671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.418525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.418823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.419093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] PATH_INFO: `/v3/projects/071ea765a80a48bca7ff3b269953fee6/users/386a769a11b44cb5aa460566e9a2ebb3/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.428952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:09.436791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071ea765a80a48bca7ff3b269953fee6, user_id=386a769a11b44cb5aa460566e9a2ebb3, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.440365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.449407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10e5d5-d73b-48b5-8373-fdcef852a45b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.457194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.458885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.459372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.460015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] PATH_INFO: `/v3/users/ea06c37647b54639a328f31b35eee7a4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.463174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea06c37647b54639a328f31b35eee7a4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.465300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.467611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 890/1770] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:09 2022] PUT /identity/v3/projects/071ea765a80a48bca7ff3b269953fee6/users/386a769a11b44cb5aa460566e9a2ebb3/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:09.472465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.474084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.515396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea06c37647b54639a328f31b35eee7a4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.516201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea06c37647b54639a328f31b35eee7a4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.529182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea06c37647b54639a328f31b35eee7a4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.536620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea06c37647b54639a328f31b35eee7a4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.537792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea06c37647b54639a328f31b35eee7a4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:09.540686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39c1e8-9fb6-4ded-af73-d91e25f84a20 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.545708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.546427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.546709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.547098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.547830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.549423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9cd0a-3f62-41dd-b9c5-e7bcc6ec951f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.550781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 881/1771] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:09 2022] DELETE /identity/v3/users/ea06c37647b54639a328f31b35eee7a4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:09.560375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 891/1772] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:09.566529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.567952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.637255 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92068cff-99d5-4f12-a18d-448edae1c018 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.642236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.643172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.643450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.644301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] PATH_INFO: `/v3/projects/071ea765a80a48bca7ff3b269953fee6/users/386a769a11b44cb5aa460566e9a2ebb3/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.646274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=071ea765a80a48bca7ff3b269953fee6, user_id=386a769a11b44cb5aa460566e9a2ebb3, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.649024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.659797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 882/1773] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:09 2022] PUT /identity/v3/projects/071ea765a80a48bca7ff3b269953fee6/users/386a769a11b44cb5aa460566e9a2ebb3/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:09.670278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.670619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.670893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.698784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] MFA Rules not processed for user `386a769a11b44cb5aa460566e9a2ebb3`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:09.701161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.754684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:09.780151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 892/1774] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:09 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:09.790894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.792592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.872864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:09.874629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:09.882313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4533e10-7d4a-418e-9b8b-7b2680c98fcc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.886849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.887514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.887805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.888083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.910542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.912426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b0084-2b5a-493d-ac6f-24ef9f87d0bb service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:09.927509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 883/1775] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:09 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:09.968603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37624-18e2-435b-83cb-e255e4517b22 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:09.983473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:09.983620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:09.983955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:09.984710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] PATH_INFO: `/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:09.986632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb2dc58a3014be98de5a2a42da6ea71)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:09.990548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.046301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb2dc58a3014be98de5a2a42da6ea71', 'request_id': 'req-a1ef84ff-2915-4160-8f98-243574203f08'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:10.053068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb2dc58a3014be98de5a2a42da6ea71', 'request_id': 'req-a1ef84ff-2915-4160-8f98-243574203f08'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:10.061303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb2dc58a3014be98de5a2a42da6ea71', 'request_id': 'req-a1ef84ff-2915-4160-8f98-243574203f08'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:10.073278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] The token cache is being invalidate because project 8bb2dc58a3014be98de5a2a42da6ea71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:10.073807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:10.074490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a1ef84ff-2915-4160-8f98-243574203f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:10.076045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 893/1776] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:09 2022] DELETE /identity/v3/projects/8bb2dc58a3014be98de5a2a42da6ea71 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:10.080840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.081168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.081446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.089013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:10.102878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:10.104099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:10.175694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 884/1777] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:10.184855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.185145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.185420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.204031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:10.205228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:10.213036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 894/1778] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:10.217047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.226640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89148022-916c-49da-9314-a9abc2f92a2b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.231126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.231767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.232049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.232330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.233043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:10.233347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.234968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.246685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 885/1779] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:10.250712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.260606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a6171-bd05-4f94-99c7-97617dc4c342 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.265101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.265749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.266035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.266311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.267037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.269014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.297166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 895/1780] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:10.301982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.312300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0a1b-77b2-4ab3-8d53-808ec67bcedb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.371080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.371786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.372073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.372348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.373026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.375272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.395492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 886/1781] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:10.400045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.411260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035c503-3c81-4d78-b3b2-19edeb13de38 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.417130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.417919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.418361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.418671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.419401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.421048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.434481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 896/1782] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:10.438895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.453184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308b1f1-5ce5-48aa-98aa-c22219d150cd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.457787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.458471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.458753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.459023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.459747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.461330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d261f14-be08-4345-a602-65a625ba711e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.472092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 887/1783] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:10.476695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.487966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e5829-d49b-4c51-aba6-35385646f1ab admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.492632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.493558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.493838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.494112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] PATH_INFO: `/v3/projects/5535588c8d7f4e82a201b8df45b7ea9f/users/8f11f0fa294a4b57955b88d1efa44011/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.511649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5535588c8d7f4e82a201b8df45b7ea9f, user_id=8f11f0fa294a4b57955b88d1efa44011, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.515122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.532041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 897/1784] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:10 2022] PUT /identity/v3/projects/5535588c8d7f4e82a201b8df45b7ea9f/users/8f11f0fa294a4b57955b88d1efa44011/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:10.543082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.543376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.543661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.571999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] MFA Rules not processed for user `8f11f0fa294a4b57955b88d1efa44011`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:10.573241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:10.643123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:10.650476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 888/1785] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:10.651719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.665856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:10.716970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9b372-7828-49db-91b4-ef368ed329f3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.721714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.722998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.723238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.723516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.724204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.726102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.750151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 898/1786] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:10.754031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.765005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d7d40-949f-44bc-bafe-d4d740fea76b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.859993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.860726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.861011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.862295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.863045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.865312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.899555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 889/1787] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:10 2022] POST /identity/v3/users => generated 300 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:10.903839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.915580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d995e-99da-44a7-b488-ec578f6cb115 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.920682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.921413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.921704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.921981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:10.922766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:10.924563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:10.942383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 899/1788] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:10.944909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:10.965681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681339-bb91-445e-b18c-808add918c24 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:10.975406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:10.980510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:10.980846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:10.981119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] PATH_INFO: `/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf/users/2a406ee8d40e40dab396e651363e06f5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.001612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3038b54e684e54a6330c9e1d13ebcf, user_id=2a406ee8d40e40dab396e651363e06f5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.004760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.020252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 890/1789] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:10 2022] PUT /identity/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf/users/2a406ee8d40e40dab396e651363e06f5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:11.024055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:11.036016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc31be-dc6e-4fd9-908c-c0c427a01dc1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:11.115026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:11.115715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.115994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.116265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.116985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.118609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064543e-8fd2-4c17-8b99-e950e3e1b2e6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.135191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 900/1790] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:11.136897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:11.155506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d92e6-dabb-462a-b639-4f32b04290ec admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:11.167242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:11.168153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.168639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.169159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] PATH_INFO: `/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf/users/2a406ee8d40e40dab396e651363e06f5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.170957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3038b54e684e54a6330c9e1d13ebcf, user_id=2a406ee8d40e40dab396e651363e06f5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.173597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.184430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 891/1791] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:11 2022] PUT /identity/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf/users/2a406ee8d40e40dab396e651363e06f5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:11.195190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.195484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.195770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.233554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] MFA Rules not processed for user `2a406ee8d40e40dab396e651363e06f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:11.234730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:11.307815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 901/1792] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:11.327321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:11.329146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:11.401818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d56d20-4874-4ed0-adea-4e4ab482ba4d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:11.405822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:11.406912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.407189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.407514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.418302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.419808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6c4ef-f672-4b9a-98f1-60b43b5bbcb8 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.442405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 892/1793] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:11.540156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:11.550573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:11.606771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d0344-3b30-472b-9a01-bdafecec89b1 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:11.611247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:11.612128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.612965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.613234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] PATH_INFO: `/v3/users/8f11f0fa294a4b57955b88d1efa44011` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.614472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f11f0fa294a4b57955b88d1efa44011)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.616407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.666371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f11f0fa294a4b57955b88d1efa44011'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:11.675604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f11f0fa294a4b57955b88d1efa44011'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:11.676706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f11f0fa294a4b57955b88d1efa44011'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:11.692847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f11f0fa294a4b57955b88d1efa44011'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:11.693592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f11f0fa294a4b57955b88d1efa44011'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:11.710537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 902/1794] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:11 2022] DELETE /identity/v3/users/8f11f0fa294a4b57955b88d1efa44011 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:11.713886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.714185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.714542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.733020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:11.734243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:11.786380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:11.800384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 893/1795] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:11.809913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:11.811692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:11.870224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6fb07-84e0-4fb2-a17d-dfb742c03dd0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:11.873798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:11.874466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:11.874748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:11.875019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:11.884003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:11.885603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:11.895138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 903/1796] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:12.205886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:12.208280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:12.227916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaca4d-ffb4-4e5f-b7b2-4a50d0fb2e15 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:12.239903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:12.240776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:12.241055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:12.241324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] PATH_INFO: `/v3/projects/5535588c8d7f4e82a201b8df45b7ea9f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:12.242571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5535588c8d7f4e82a201b8df45b7ea9f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:12.248075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:12.295225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5535588c8d7f4e82a201b8df45b7ea9f', 'request_id': 'req-91642282-454e-4552-8a12-fac43427f710'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.302689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5535588c8d7f4e82a201b8df45b7ea9f', 'request_id': 'req-91642282-454e-4552-8a12-fac43427f710'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.309150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5535588c8d7f4e82a201b8df45b7ea9f', 'request_id': 'req-91642282-454e-4552-8a12-fac43427f710'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.324322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] The token cache is being invalidate because project 5535588c8d7f4e82a201b8df45b7ea9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:12.324618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.325411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.327393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 894/1797] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:12 2022] DELETE /identity/v3/projects/5535588c8d7f4e82a201b8df45b7ea9f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:12.331823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:12.343720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a325b-8535-42f6-847b-33683b3b69c9 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:12.375997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:12.412475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:12.413355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:12.413644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:12.413923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] PATH_INFO: `/v3/users/2a406ee8d40e40dab396e651363e06f5` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:12.415147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a406ee8d40e40dab396e651363e06f5)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:12.417084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:12.461438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a406ee8d40e40dab396e651363e06f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.468380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a406ee8d40e40dab396e651363e06f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.469034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a406ee8d40e40dab396e651363e06f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.481590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a406ee8d40e40dab396e651363e06f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.482231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a406ee8d40e40dab396e651363e06f5'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.494254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 904/1798] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:12 2022] DELETE /identity/v3/users/2a406ee8d40e40dab396e651363e06f5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:12.779746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:12.781240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:12.794130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91642282-454e-4552-8a12-fac43427f710 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:12.867485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:12.868353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:12.868637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:12.868910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] PATH_INFO: `/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:12.870088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3038b54e684e54a6330c9e1d13ebcf)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:12.872080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:12.922245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3038b54e684e54a6330c9e1d13ebcf', 'request_id': 'req-769b7a73-a9bb-47b3-b4d1-91e06b36655e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.929204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3038b54e684e54a6330c9e1d13ebcf', 'request_id': 'req-769b7a73-a9bb-47b3-b4d1-91e06b36655e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.935387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3038b54e684e54a6330c9e1d13ebcf', 'request_id': 'req-769b7a73-a9bb-47b3-b4d1-91e06b36655e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.953094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:12.953258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] The token cache is being invalidate because project ba3038b54e684e54a6330c9e1d13ebcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:12.953530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.958976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-769b7a73-a9bb-47b3-b4d1-91e06b36655e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:12.959396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:12.959487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 895/1799] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:12 2022] DELETE /identity/v3/projects/ba3038b54e684e54a6330c9e1d13ebcf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:12.961134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858eacf-dd38-4365-8b8f-1ff8e21a42f7 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:12.961583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858eacf-dd38-4365-8b8f-1ff8e21a42f7 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:12.962076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858eacf-dd38-4365-8b8f-1ff8e21a42f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:12.978835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f77e1-9d5d-4a3a-8e8e-504c3cad4027 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:12.982834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c858eacf-dd38-4365-8b8f-1ff8e21a42f7 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:12.984383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c858eacf-dd38-4365-8b8f-1ff8e21a42f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:12.999281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:13.077116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 896/1800] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:13.078233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.079141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.079432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.079705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] PATH_INFO: `/v3/users/bc8cd03051a64d17bccb5b5bdaa45898` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.080992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8cd03051a64d17bccb5b5bdaa45898)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.083026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.085987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.086486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.086763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.115048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:13.116720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:13.145262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 897/1801] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:13.150278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.161687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4bc27-cc40-4da6-9264-c3a1acaa529b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.169696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.170400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.170688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.170972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.171507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8cd03051a64d17bccb5b5bdaa45898'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:13.171755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:13.172070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.173699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.183608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:13.194174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8cd03051a64d17bccb5b5bdaa45898'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:13.195207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8cd03051a64d17bccb5b5bdaa45898'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:13.219893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 898/1802] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:13.225011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.234921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8cd03051a64d17bccb5b5bdaa45898'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:13.235791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2563226f-98c2-4440-93f0-6132f6473e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8cd03051a64d17bccb5b5bdaa45898'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:13.237901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f75a2-e504-4c1f-9e67-3bdfb4bdd940 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.243714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.244407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.244688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.244966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.245673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.247616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.252763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 905/1803] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:12 2022] DELETE /identity/v3/users/bc8cd03051a64d17bccb5b5bdaa45898 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:13.261696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.262010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.262300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.295473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 899/1804] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:13.298841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.303179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:13.312910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:13.314160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:13.391932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869999d2-76e6-4cd4-b3a5-d161ca8737c6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.397962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.398928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.398928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.399219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.400449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.402748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.422430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 906/1805] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:13.433799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 900/1806] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:13.433971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.434960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.440300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:13.454864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0643-3964-4544-83b8-3f477214b525 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.493489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.495068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.496927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.497565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.498875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.504045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.535599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 901/1807] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:13.540894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.544633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b613-97e1-475e-8df4-15463e79f06a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.549524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.550885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.551215 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.553010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.571262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cddc7-a133-4c86-b70f-27b41cfebe71 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.579727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.585640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.587327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.588006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.588290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.588868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.589635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.594042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4c9e9-8dc5-45df-b677-69675b760951 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.609238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 907/1808] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:13.618547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 902/1809] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:13.620220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:13.637027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d8865-12fd-418c-8693-375daf70eaeb service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:13.643291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:13.644461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.644944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.645209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] PATH_INFO: `/v3/projects/477777272f704e0cbb45d995e8fdbcb3/users/52cf8244c84b4907ae978f73b6f70230/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.666417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] RBAC: Authorizing `identity:create_grant(project_id=477777272f704e0cbb45d995e8fdbcb3, user_id=52cf8244c84b4907ae978f73b6f70230, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:13.671106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:13.700963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 908/1810] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:13 2022] PUT /identity/v3/projects/477777272f704e0cbb45d995e8fdbcb3/users/52cf8244c84b4907ae978f73b6f70230/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:13.713121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:13.713512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:13.713796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:13.774364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] MFA Rules not processed for user `52cf8244c84b4907ae978f73b6f70230`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:13.777445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:13.829982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:13.944284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 903/1811] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:13.949375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.066683 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:14.096980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3310e44-dfaf-4f16-94de-d9577c4e762b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.112757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:14.113771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.119120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.119311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.120094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:14.122103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:14.192457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 909/1812] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:13 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:14.198040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.216479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00c0a2-451f-4379-a64e-75d2d989adb0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.227598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.229955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:14.332848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:14.333603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.333893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.334170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.334896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:14.338857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:14.381517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 904/1813] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:14 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:14.382818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.397215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219723d-1bd3-4c7b-a17a-2a9d7e903255 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.403716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:14.407088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.408184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.408811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.414662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:14.418207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:14.441824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a89f4-c3d6-476a-acec-acd35d095857 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.455337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:14.456411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.456533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.456809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] PATH_INFO: `/v3/projects/65e6f580e58644e5a9a95ddc572212e9` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.458381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65e6f580e58644e5a9a95ddc572212e9)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:14.459529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 905/1814] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:14.464699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.465770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:14.517041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646feaa-6666-43cd-9a95-139587d2057d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.532379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:14.537154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.539094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.539542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] PATH_INFO: `/v3/projects/2e0946a4b08a4831b27f7e3b57220c17/users/a6a89fbf1d174b8eb06e3f4cf0d963f5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.599652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0946a4b08a4831b27f7e3b57220c17, user_id=a6a89fbf1d174b8eb06e3f4cf0d963f5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:14.600804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65e6f580e58644e5a9a95ddc572212e9', 'request_id': 'req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:14.603722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110a5e4-25bf-4aa4-98cc-968dbebbb84d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:14.633982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65e6f580e58644e5a9a95ddc572212e9', 'request_id': 'req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:14.636953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 906/1815] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:14 2022] PUT /identity/v3/projects/2e0946a4b08a4831b27f7e3b57220c17/users/a6a89fbf1d174b8eb06e3f4cf0d963f5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:14.641369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:14.641669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:14.643120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:14.655433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65e6f580e58644e5a9a95ddc572212e9', 'request_id': 'req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:14.668919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:14.670521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:14.688022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] The token cache is being invalidate because project 65e6f580e58644e5a9a95ddc572212e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:14.688617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:14.691160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:14.695290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 910/1816] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:14 2022] DELETE /identity/v3/projects/65e6f580e58644e5a9a95ddc572212e9 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:14.696715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.700612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:14.812597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 907/1817] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 30 17:53:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:14.818597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:14.823275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:14.918584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f70b1-4759-4f0c-89ff-ff33fd81d2b8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:14.920760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:14.939040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:14.998069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83290376-2246-41b8-b90e-a3bfc0029016 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:15.013161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:15.014046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.014343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.014619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] PATH_INFO: `/v3/users/386a769a11b44cb5aa460566e9a2ebb3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.016035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=386a769a11b44cb5aa460566e9a2ebb3)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:15.017975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:15.068321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:15.069144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.069444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.069699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.070482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:15.072168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:15.107805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 911/1818] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:15.108671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:15.117394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.154548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c32f5-1874-40f9-ae0e-b97ed71e485b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:15.159740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386a769a11b44cb5aa460566e9a2ebb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.160820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386a769a11b44cb5aa460566e9a2ebb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.173893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:15.175272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.175562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.175879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.178459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.195456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:15.196985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:15.202721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386a769a11b44cb5aa460566e9a2ebb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.211557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 912/1819] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:15.212847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:15.216207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.220369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386a769a11b44cb5aa460566e9a2ebb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.222765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386a769a11b44cb5aa460566e9a2ebb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.255717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 908/1820] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:14 2022] DELETE /identity/v3/users/386a769a11b44cb5aa460566e9a2ebb3 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:15.257206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:15.258878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.292748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8ef99-6b11-4ea4-9a0a-aae6f626ca09 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:15.426889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38f2b-dbc1-4607-887d-6a3d18b00700 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:15.441295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:15.445728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.446364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.446364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] PATH_INFO: `/v3/users/2b46599ea79840b4869d4b1086b3ae25` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.447652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:15.449963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.450622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.450969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b46599ea79840b4869d4b1086b3ae25)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:15.457605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:15.465199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] PATH_INFO: `/v3/projects/2e0946a4b08a4831b27f7e3b57220c17/users/a6a89fbf1d174b8eb06e3f4cf0d963f5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.473349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0946a4b08a4831b27f7e3b57220c17, user_id=a6a89fbf1d174b8eb06e3f4cf0d963f5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:15.479493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4289f2-ddbc-4aa8-8126-1a5f4b6dc8c2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:15.512734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 909/1821] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:15 2022] PUT /identity/v3/projects/2e0946a4b08a4831b27f7e3b57220c17/users/a6a89fbf1d174b8eb06e3f4cf0d963f5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:15.526208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.527448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.528199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.609774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b46599ea79840b4869d4b1086b3ae25'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.646860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] MFA Rules not processed for user `a6a89fbf1d174b8eb06e3f4cf0d963f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:15.650143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.650694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b46599ea79840b4869d4b1086b3ae25'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.651460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b46599ea79840b4869d4b1086b3ae25'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.727188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b46599ea79840b4869d4b1086b3ae25'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.734998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-881f1ebb-91e7-4054-9a4f-1feead73310a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b46599ea79840b4869d4b1086b3ae25'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:15.806722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 913/1822] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:15 2022] DELETE /identity/v3/users/2b46599ea79840b4869d4b1086b3ae25 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:15.807858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:15.812160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:15.812756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:15.816063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:15.856382 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:15.858122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.869404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:15.920238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 910/1823] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:15.929444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:15.931291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:15.967065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 914/1824] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:16.073974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:16.075905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:16.117773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a2ff-5416-486d-b501-e381ebda11b2 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:16.276988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8e27-52ca-4e7a-a3ce-09290701723c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:16.281772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:16.282807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:16.283100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:16.283379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:16.308003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:16.310181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:16.326362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 915/1825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:16.327269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:16.329422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:16.351778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:16.352650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:16.353093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:16.353379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:16.354907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99591a4-a0a7-42e0-88ea-7930a67730c8 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:16.363235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:16.364107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:16.364389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:16.375703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] PATH_INFO: `/v3/users/51c4ccac97d248f3af95d3beaddab009` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:16.378096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c4ccac97d248f3af95d3beaddab009)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:16.380644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:16.473779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c4ccac97d248f3af95d3beaddab009'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:16.500538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c4ccac97d248f3af95d3beaddab009'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:16.502013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c4ccac97d248f3af95d3beaddab009'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:16.540317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c4ccac97d248f3af95d3beaddab009'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:16.540895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d898a391-142c-49fd-b7bf-b97c7e07be3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c4ccac97d248f3af95d3beaddab009'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:16.583199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 916/1826] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:16 2022] DELETE /identity/v3/users/51c4ccac97d248f3af95d3beaddab009 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:16.583471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:16.589346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0ed74-6e26-4dae-83f6-3b380f2f65cf service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:16.591723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:16.593601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:16.598073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:16.610659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 911/1827] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:16.664117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:16.668265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:16.733618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:16.844167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 917/1828] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:17.180131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.181285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6d19-bb3a-4f92-8bfc-996a9cffb86b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.182372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:17.186839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6d19-bb3a-4f92-8bfc-996a9cffb86b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.187724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c6d19-bb3a-4f92-8bfc-996a9cffb86b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.193302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 912/1829] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:53:17 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:53:17.213210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.213504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.213785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.318467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:17.323150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:17.388157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfc55a-cb96-4fdb-86ab-1656fc64a650 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:17.392775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:17.394300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.394763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.395286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.414276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:17.416175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:17.421100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:17.431374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 918/1830] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:17.432464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.434302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:17.518954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e80011-4cab-4bf0-92a8-1b60ace93c4f service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:17.524619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a89fbf1d174b8eb06e3f4cf0d963f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e0946a4b08a4831b27f7e3b57220c17', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:17.525561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.526831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.527243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.528091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:17.529283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:17.534568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:17.562540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 913/1831] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:17.570322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 919/1832] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:17.574747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.609857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626d6b6-3471-4c2d-9396-ce3ba74b3f3f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:17.626795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a89fbf1d174b8eb06e3f4cf0d963f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e0946a4b08a4831b27f7e3b57220c17', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:17.628621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.629171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.629583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.630549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:17.633047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:17.694767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 914/1833] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:17 2022] POST /identity/v3/projects => generated 354 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:17.696152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.713591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120dade-b92c-4ab4-b62c-4fe42a12a098 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:17.784701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.794766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:17.910118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a89fbf1d174b8eb06e3f4cf0d963f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e0946a4b08a4831b27f7e3b57220c17', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:17.911051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:17.911598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:17.912039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:17.912884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:17.913331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:17.916211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:17.931167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:17.937039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 920/1834] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:17.938177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:17.942225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:18.035543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d595d8-ff74-4617-a277-4c6a6036251c tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.048896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.050352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.050653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.050927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] PATH_INFO: `/v3/projects/071ea765a80a48bca7ff3b269953fee6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.059522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff8de7-9a31-418f-91fc-fd04b7b790a6 tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.059717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=071ea765a80a48bca7ff3b269953fee6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.061955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:18.075356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.076448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.078879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.079501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.100493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.102514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:18.122563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 921/1835] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:18.124471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:18.146701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021516e-4b32-474a-8c77-040cce2949c7 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.154874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6a89fbf1d174b8eb06e3f4cf0d963f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e0946a4b08a4831b27f7e3b57220c17', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.156461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.156952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.157571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] PATH_INFO: `/v3/projects/b2bb160431ca461daefa9a43084a9dbe` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.162383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b2bb160431ca461daefa9a43084a9dbe)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.165740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:18.249803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071ea765a80a48bca7ff3b269953fee6', 'request_id': 'req-444614d1-0070-475b-aecf-e6bcdaab2e3b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.276806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071ea765a80a48bca7ff3b269953fee6', 'request_id': 'req-444614d1-0070-475b-aecf-e6bcdaab2e3b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.294908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2bb160431ca461daefa9a43084a9dbe', 'request_id': 'req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.301797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071ea765a80a48bca7ff3b269953fee6', 'request_id': 'req-444614d1-0070-475b-aecf-e6bcdaab2e3b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.320748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2bb160431ca461daefa9a43084a9dbe', 'request_id': 'req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.348258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] The token cache is being invalidate because project 071ea765a80a48bca7ff3b269953fee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:18.348602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.349420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-444614d1-0070-475b-aecf-e6bcdaab2e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.354826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2bb160431ca461daefa9a43084a9dbe', 'request_id': 'req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.364361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 915/1836] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:17 2022] DELETE /identity/v3/projects/071ea765a80a48bca7ff3b269953fee6 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:18.372335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.373163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.373163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.402227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] The token cache is being invalidate because project b2bb160431ca461daefa9a43084a9dbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:18.402832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.407616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c62e39e8-3eed-4dcc-a9ef-973eb73eec9e tempest-VolumeQuotaClassesTest-1663278326 tempest-VolumeQuotaClassesTest-1663278326-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:18.415125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 922/1837] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:18 2022] DELETE /identity/v3/projects/b2bb160431ca461daefa9a43084a9dbe => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:18.449547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:18.461895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:18.512738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:18.626784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 916/1838] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:18.646952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.646952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.646952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.678719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:18.679695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:18.719157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:18.727646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 923/1839] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:18.733706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e30945-0255-4317-8db2-ce77d33d64e3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.754355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:18.755178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:18.764750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.765356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.765634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.765902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.776259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:18.776578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.785233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:18.793074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623ad6b-8707-4c02-b38f-011076d0a403 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.805128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.806361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.806507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.806741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] PATH_INFO: `/v3/projects/a3a247f5863144dba2376d37aea80c08` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.816743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a247f5863144dba2376d37aea80c08)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.831935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:18.832978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 917/1840] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:18.833962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:18.918985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8783-d2b7-490b-b594-cfd4aba2ba7c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:18.943561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:18.944337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:18.944661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:18.944986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:18.948895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:18.951150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:19.072848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a247f5863144dba2376d37aea80c08', 'request_id': 'req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.074418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 918/1841] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:18 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:19.075414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.087235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:19.089603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a247f5863144dba2376d37aea80c08', 'request_id': 'req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.108266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a247f5863144dba2376d37aea80c08', 'request_id': 'req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.137637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] The token cache is being invalidate because project a3a247f5863144dba2376d37aea80c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:19.137932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.142081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0723a60c-3bc6-492c-9a17-6b2f2cf4932b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.146996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 924/1842] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:18 2022] DELETE /identity/v3/projects/a3a247f5863144dba2376d37aea80c08 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:19.155002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.163983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.164307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.223875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:19.227183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:19.288189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faba2c-620c-4186-b5e8-9ccc8ce2c963 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:19.303227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:19.304007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.304319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.304639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.305340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:19.311385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:19.391065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 919/1843] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:19 2022] POST /identity/v3/users => generated 287 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:19.392189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.394166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:19.430832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3283a-1d2c-4a78-8a2f-5b932eaf7ce3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:19.438772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 925/1844] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:19.439734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.447795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:19.448924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:19.449884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.450104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.450418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] PATH_INFO: `/v3/users/52cf8244c84b4907ae978f73b6f70230` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.453772 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52cf8244c84b4907ae978f73b6f70230)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:19.463174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:19.489022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c16a9-a169-47d7-a1d6-fc23d7e0c3c8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:19.505741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:19.507860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.508129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.508430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.509145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:19.511565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaccf43-8333-47da-b8a7-533dd89901cb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:19.543920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 926/1845] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:19.545862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.546549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.546729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.607398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:19.625066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:19.669720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 927/1846] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:19.672414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.714558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d18c2-2435-4bca-b3af-8b656abd3ddb None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:19.720874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52cf8244c84b4907ae978f73b6f70230'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.723776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52cf8244c84b4907ae978f73b6f70230'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.742544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:19.744027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:19.745121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:19.745637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:19.746844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:19.750205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:19.777737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52cf8244c84b4907ae978f73b6f70230'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.780554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 928/1847] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:19.786021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.798218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52cf8244c84b4907ae978f73b6f70230'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.799408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52cf8244c84b4907ae978f73b6f70230'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:19.800086 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:19.825780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:19.833896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34168e3-cb5e-40b9-b76f-3711bbfc0370 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:19.837245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 920/1848] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:19 2022] DELETE /identity/v3/users/52cf8244c84b4907ae978f73b6f70230 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:19.837245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:19.841929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.028855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.029594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.029885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.030160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.030999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:20.031335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:20.033060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c866-c9d3-40ae-a81f-200e01a0960e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:20.062719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edcdb6-7097-4835-8e8d-6b4bf4ff4738 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:20.069841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.074538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.074538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.074538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] PATH_INFO: `/v3/projects/5e499f12cd36471bbbf9879a5e5af187/users/aa1e43ff2a414d9f8df750917bac5865/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.075244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 929/1849] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:20.077172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.077580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.077781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.094568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e499f12cd36471bbbf9879a5e5af187, user_id=aa1e43ff2a414d9f8df750917bac5865, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:20.098030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:20.118777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:20.124425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.132877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 921/1850] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:19 2022] PUT /identity/v3/projects/5e499f12cd36471bbbf9879a5e5af187/users/aa1e43ff2a414d9f8df750917bac5865/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:20.135646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:20.151473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.259100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:20.321124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 930/1851] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:20.322212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:20.336954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f62c-8898-41e9-af5b-73c1140011c0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:20.342056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.342994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.343189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.343461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.344163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:20.346934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd549-9c65-4ac6-99b1-df1b2d9c7e28 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:20.394487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f7de-4205-4fe4-894d-0a561d039859 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:20.396317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 931/1852] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:20 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:20.398428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.398738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.399021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.473871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] MFA Rules not processed for user `aa1e43ff2a414d9f8df750917bac5865`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:20.476071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.538682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.539809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.540264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.540710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] PATH_INFO: `/v3/projects/88c89834908c41dcbbd2d784d033215d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.544466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c89834908c41dcbbd2d784d033215d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:20.546780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:20.635449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:20.640308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 932/1853] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:20.644775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:20.647808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.657832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c89834908c41dcbbd2d784d033215d', 'request_id': 'req-afa44487-68d6-4f15-90b7-fa32714e0ae4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:20.667534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c89834908c41dcbbd2d784d033215d', 'request_id': 'req-afa44487-68d6-4f15-90b7-fa32714e0ae4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:20.675703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c89834908c41dcbbd2d784d033215d', 'request_id': 'req-afa44487-68d6-4f15-90b7-fa32714e0ae4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:20.725131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] The token cache is being invalidate because project 88c89834908c41dcbbd2d784d033215d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:20.725848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:20.729082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:20.742139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 922/1854] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:20 2022] DELETE /identity/v3/projects/88c89834908c41dcbbd2d784d033215d => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:20.742929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:20.745656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.824486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875a4e7-74db-4bcb-8bef-8cacfab9fc82 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:20.944519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa44487-68d6-4f15-90b7-fa32714e0ae4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:20.950442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.951110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.951409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.951676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.959908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:20.962998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:20.964267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:20.964555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:20.964871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:20.965572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:20.968503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:20.999158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.002747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.025637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 923/1855] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:20 2022] POST /identity/v3/users => generated 284 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:21.026531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.029277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 933/1856] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:21.030131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.031595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:21.043052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:21.081565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad96543-dee4-4ae8-bfdc-1154ca0dc734 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.082985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa60879-4a3b-4565-8239-4dd5b2108370 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.093569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.095497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.095497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.095703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.096841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.098740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.099635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.099919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.100195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] PATH_INFO: `/v3/users/9a34e2293aec48f9a85b3edd60ad3bb0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.102642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a34e2293aec48f9a85b3edd60ad3bb0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.104722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.107363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.140495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:21.152244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 924/1857] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:21 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:21.154668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.248466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a34e2293aec48f9a85b3edd60ad3bb0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:21.272688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a34e2293aec48f9a85b3edd60ad3bb0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:21.273449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a34e2293aec48f9a85b3edd60ad3bb0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:21.306617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a34e2293aec48f9a85b3edd60ad3bb0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:21.307373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a34e2293aec48f9a85b3edd60ad3bb0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:21.308159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19fab5-693d-4bcc-a6ee-d0781844f241 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.322757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.323712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.324000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.324279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.325379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.330090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.341498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 934/1858] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:21 2022] DELETE /identity/v3/users/9a34e2293aec48f9a85b3edd60ad3bb0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:21.342400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.344048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:21.358838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 925/1859] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:21.362952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.378607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:21.419986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807325e8-4095-417c-b7a8-734639f1443a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.533566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:21.579054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3471049-7a29-4605-ae59-95609267310a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.591422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.592162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.592447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.592722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.595770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.597470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.630126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.630854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.631130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.631402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.632330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.634689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.646834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 926/1860] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:21.662706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 935/1861] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:21 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:21.664640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.665649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.684274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950097d-8dff-425e-9e97-e48d74057689 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.695470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.696342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.696737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.697162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.697968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.700329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.701690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6050d-586a-4cc0-9a1a-357ba29b280f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.714964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.717651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.718676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.719277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] PATH_INFO: `/v3/projects/17cef16b0169473ca88243c07c811cc0/users/af3ab8b1dced46f2a164ea3f35de80bf/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.721920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 936/1862] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:21.723990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.755394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32421a75-dabb-4ecf-a053-2ff3c4a60c03 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:21.773034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:21.776362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.777992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.778367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] PATH_INFO: `/v3/projects/54206c45a873429a971e1908f734026f/users/ea1ae81cc1dd4b3394396c2830741e9a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.805905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cef16b0169473ca88243c07c811cc0, user_id=af3ab8b1dced46f2a164ea3f35de80bf, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.815858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a08e5-f49f-43c1-846f-b511ff7a6327 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.827346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54206c45a873429a971e1908f734026f, user_id=ea1ae81cc1dd4b3394396c2830741e9a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:21.843723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:21.853850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 927/1863] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:21 2022] PUT /identity/v3/projects/17cef16b0169473ca88243c07c811cc0/users/af3ab8b1dced46f2a164ea3f35de80bf/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:21.876685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:21.877067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:21.877885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:21.909361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 937/1864] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:21 2022] PUT /identity/v3/projects/54206c45a873429a971e1908f734026f/users/ea1ae81cc1dd4b3394396c2830741e9a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:21.915047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:21.981625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] MFA Rules not processed for user `af3ab8b1dced46f2a164ea3f35de80bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:21.983511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.103016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:22.139090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c115ff-5639-4a5d-8d98-26a91b5cb970 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.144676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.145582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.146209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.146689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.148415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.151720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.155386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 928/1865] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:22.155895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.167647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 938/1866] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:22.173655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.176753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c9ac6-7feb-4a46-a8e0-822a6fa1071c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.185220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.185220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.185220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.185220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.185678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.188588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.189228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b8c5-d053-4fa9-9227-8cf619bcc9c0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.196589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.197795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.198080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.198645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] PATH_INFO: `/v3/projects/54206c45a873429a971e1908f734026f/users/ea1ae81cc1dd4b3394396c2830741e9a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.201417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54206c45a873429a971e1908f734026f, user_id=ea1ae81cc1dd4b3394396c2830741e9a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.204538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2813bb-e740-4b2d-bdb1-324ee86f2c54 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.219441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 939/1867] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:22 2022] PUT /identity/v3/projects/54206c45a873429a971e1908f734026f/users/ea1ae81cc1dd4b3394396c2830741e9a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:22.231012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.231842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.232720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.237561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 929/1868] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:22 2022] POST /identity/v3/projects => generated 369 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:22.237561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.239787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.317929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] MFA Rules not processed for user `ea1ae81cc1dd4b3394396c2830741e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:22.322287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.374642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d082e2b-933c-4619-8539-50447b4b2e3c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.381549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.382453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.382741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.383267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] PATH_INFO: `/v3/projects/477777272f704e0cbb45d995e8fdbcb3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.384626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=477777272f704e0cbb45d995e8fdbcb3)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.387355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.397104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:22.450581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 940/1869] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:22.451593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.469297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f49ff-ede9-41e9-a2d6-1ab8761b0c16 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.491425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477777272f704e0cbb45d995e8fdbcb3', 'request_id': 'req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.500847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477777272f704e0cbb45d995e8fdbcb3', 'request_id': 'req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.517348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477777272f704e0cbb45d995e8fdbcb3', 'request_id': 'req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.541162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] The token cache is being invalidate because project 477777272f704e0cbb45d995e8fdbcb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:22.541932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.543180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.546033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 930/1870] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:22 2022] DELETE /identity/v3/projects/477777272f704e0cbb45d995e8fdbcb3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:22.547272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.550429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.554563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.555433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.555975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.556319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.557489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.562182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.568775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416c9b1-5a65-4f26-964f-2ecf43ee1aaf admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.575254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.576201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.576692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.576798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] PATH_INFO: `/v3/projects/4edc9081ae114fce88794185f5f099dd` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.578103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edc9081ae114fce88794185f5f099dd)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.580316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.590899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 941/1871] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:22 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:22.591940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.594843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.655964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:22.659619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edc9081ae114fce88794185f5f099dd', 'request_id': 'req-66f5eeae-1d25-4731-8367-b7247d2d378c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.672280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edc9081ae114fce88794185f5f099dd', 'request_id': 'req-66f5eeae-1d25-4731-8367-b7247d2d378c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.683062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204a65-2fb4-407f-91e8-29c0024dbafb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.686973 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edc9081ae114fce88794185f5f099dd', 'request_id': 'req-66f5eeae-1d25-4731-8367-b7247d2d378c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.709697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] The token cache is being invalidate because project 4edc9081ae114fce88794185f5f099dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:22.710536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.711516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.719002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 931/1872] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:22 2022] DELETE /identity/v3/projects/4edc9081ae114fce88794185f5f099dd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:22.719865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.721858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.758037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.758805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.759161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.759514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.761896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.808110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5eeae-1d25-4731-8367-b7247d2d378c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.816942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.818176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.818670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.819684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] PATH_INFO: `/v3/users/a6a89fbf1d174b8eb06e3f4cf0d963f5` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.821871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a89fbf1d174b8eb06e3f4cf0d963f5)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.824572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.887233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:22.889251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.891797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.920556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 942/1873] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:22.923062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:22.925290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:22.949918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a89fbf1d174b8eb06e3f4cf0d963f5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.950761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a89fbf1d174b8eb06e3f4cf0d963f5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.959520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684ae95-30c7-46a7-918b-910bdfb32239 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:22.965915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:22.967011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.968812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.969192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:22.970235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:22.972860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ed6b0-7d9b-498b-a998-6995e4432cab admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:22.982748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a89fbf1d174b8eb06e3f4cf0d963f5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:22.990843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 943/1874] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:22.995059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655341-507c-4545-9c84-0fdcd8edab28 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:22.996008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655341-507c-4545-9c84-0fdcd8edab28 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:22.996008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655341-507c-4545-9c84-0fdcd8edab28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.008261 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a89fbf1d174b8eb06e3f4cf0d963f5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:23.009234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a89fbf1d174b8eb06e3f4cf0d963f5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:23.045353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 932/1875] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:22 2022] DELETE /identity/v3/users/a6a89fbf1d174b8eb06e3f4cf0d963f5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:23.047692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.050450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.053131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-5f655341-507c-4545-9c84-0fdcd8edab28 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:23.054476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5f655341-507c-4545-9c84-0fdcd8edab28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.221789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 944/1876] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:23.233009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572833db-66bc-413e-9c86-bdf0fa5f9e5a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.233322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572833db-66bc-413e-9c86-bdf0fa5f9e5a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.234772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572833db-66bc-413e-9c86-bdf0fa5f9e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.281228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-572833db-66bc-413e-9c86-bdf0fa5f9e5a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:23.283797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-572833db-66bc-413e-9c86-bdf0fa5f9e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.293200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820e9e2-cd22-4ae4-8f21-628a083f6ddd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.307236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.307919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 945/1877] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:23.308888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.309021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519035b-de71-4025-853a-ca6ea7fb13e5 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.309320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519035b-de71-4025-853a-ca6ea7fb13e5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.309420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.309624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519035b-de71-4025-853a-ca6ea7fb13e5 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.309760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] PATH_INFO: `/v3/projects/ce12e97571f94917a5d4e866496aacb3/users/17c9b0923800467380122f3564017991/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.314069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 946/1878] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:53:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:53:23.315313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519035b-de71-4025-853a-ca6ea7fb13e5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.333962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519035b-de71-4025-853a-ca6ea7fb13e5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.354812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.355490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.355778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.356052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.358323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:23.358323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.359464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce12e97571f94917a5d4e866496aacb3, user_id=17c9b0923800467380122f3564017991, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.359835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1950db-494f-4fa1-9ea3-369c934f1c39 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.365621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.414394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 947/1879] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:23.427282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.427282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.427282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.431862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 933/1880] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:23 2022] PUT /identity/v3/projects/ce12e97571f94917a5d4e866496aacb3/users/17c9b0923800467380122f3564017991/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:23.432347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.517963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:23.520433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.544310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec593a67-1e34-4bd2-9329-8fc41db16a6f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.564155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:23.570093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.571281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.571573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.571856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.572563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.574819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.643858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 934/1881] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:23 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:23.644750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.662260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:23.732296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 948/1882] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:53:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:23.733332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.769241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d6318-d038-4f95-ada1-f2d86d8ec1c5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.792223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484f07d-351f-43b4-a97a-8f29437690d1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.807105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.808010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.808372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.808909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.809521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.811759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.829623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 935/1883] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:23.830714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.833526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.875986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.876846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.877050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.878553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.879480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.883756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.918147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 949/1884] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:23 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:23.919048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:23.921410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:23.936685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70062e-02bb-4e4a-bd3c-7d7b70045d13 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.945280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.946411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91bfd9-719c-4913-83e2-1d5262123e99 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:23.947055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.947641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.948152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] PATH_INFO: `/v3/users/aa1e43ff2a414d9f8df750917bac5865` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.950838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1e43ff2a414d9f8df750917bac5865)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.953465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.955012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:23.955875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:23.956305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:23.956771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:23.968263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:23.972398 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:23.991041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 936/1885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:23.999524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.031788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70738-b1d4-4042-adcb-278754208657 service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.040913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.042073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.042391 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.042721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] PATH_INFO: `/v3/projects/ce12e97571f94917a5d4e866496aacb3/users/17c9b0923800467380122f3564017991/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.044779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce12e97571f94917a5d4e866496aacb3, user_id=17c9b0923800467380122f3564017991, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.047746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.055986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1e43ff2a414d9f8df750917bac5865'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.060265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 937/1886] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:23 2022] PUT /identity/v3/projects/ce12e97571f94917a5d4e866496aacb3/users/17c9b0923800467380122f3564017991/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:24.062721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.068081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1e43ff2a414d9f8df750917bac5865'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.068808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1e43ff2a414d9f8df750917bac5865'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.086287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1e43ff2a414d9f8df750917bac5865'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.087512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1e43ff2a414d9f8df750917bac5865'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.103216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 950/1887] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:23 2022] DELETE /identity/v3/users/aa1e43ff2a414d9f8df750917bac5865 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:24.107651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.109082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.124413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:24.153341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb27464-0128-451d-a104-8b68f6526394 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.172342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:24.186170 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426889a-533d-409b-afa2-2f227695a30c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.194832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.195626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.195914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.196190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] PATH_INFO: `/v3/projects/2e0946a4b08a4831b27f7e3b57220c17` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.197562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0946a4b08a4831b27f7e3b57220c17)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.199978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.225231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.226095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.226418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.226702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.227445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.229090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef96a0-7deb-4dcb-a605-4255d0de285a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.240576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 938/1888] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:24.244801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.245302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.245760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.255859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0946a4b08a4831b27f7e3b57220c17', 'request_id': 'req-b79878b0-bf34-4067-b597-b5ddbf621af7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.262044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0946a4b08a4831b27f7e3b57220c17', 'request_id': 'req-b79878b0-bf34-4067-b597-b5ddbf621af7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.271740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0946a4b08a4831b27f7e3b57220c17', 'request_id': 'req-b79878b0-bf34-4067-b597-b5ddbf621af7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.278524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] MFA Rules not processed for user `17c9b0923800467380122f3564017991`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:24.280254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.286074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] The token cache is being invalidate because project 2e0946a4b08a4831b27f7e3b57220c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:24.286419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.287543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b79878b0-bf34-4067-b597-b5ddbf621af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:24.292854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 951/1889] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:24 2022] DELETE /identity/v3/projects/2e0946a4b08a4831b27f7e3b57220c17 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:24.294373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6335-6f94-442f-b6db-2b586a806ff5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.294671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6335-6f94-442f-b6db-2b586a806ff5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.295109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6335-6f94-442f-b6db-2b586a806ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.315239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b2ad6335-6f94-442f-b6db-2b586a806ff5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:24.316550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b2ad6335-6f94-442f-b6db-2b586a806ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.367737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 939/1890] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:24.369157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.370692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.399374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 952/1891] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:24.400675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.400970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.401253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.422592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:24.423825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.433265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 953/1892] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:24.434079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.439365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.459049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dec8f-4579-4681-a2a7-d87d3eb291e0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.466931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.468594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.469110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.470085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.472469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.474618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.485779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:24.497820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 940/1893] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:24.499763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.502165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.514040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52a7ac-5b09-4aac-9e38-cb8d9825aaa5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.519158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.519994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.520254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.520622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.532767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.534911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bbd54-9da5-4cc4-a2fd-5390599c25f6 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.547308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 954/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:24 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:24.548480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535362-f369-4390-9b63-dd912935ca8f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.548774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535362-f369-4390-9b63-dd912935ca8f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.549033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58535362-f369-4390-9b63-dd912935ca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.576132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-58535362-f369-4390-9b63-dd912935ca8f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:24.577367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-58535362-f369-4390-9b63-dd912935ca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.580218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:24.589733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 955/1895] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:24.590576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58535362-f369-4390-9b63-dd912935ca8f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.607587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58535362-f369-4390-9b63-dd912935ca8f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.612770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39156-d50e-4843-9187-feacfb4b427d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.613390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.614351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.614671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.615032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] PATH_INFO: `/v3/projects/b85465d60267462e98351b6154d8e1a7/users/03bf69013f46445ca3d81482dbb7e5e5/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.617682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.629558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.629824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.630102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.641990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.644281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.658343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b85465d60267462e98351b6154d8e1a7, user_id=03bf69013f46445ca3d81482dbb7e5e5, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.660124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 941/1896] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:24.661850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7b6d4-5a5e-4da2-b0d9-de03a14c5b30 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.663992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.695248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2712-cf13-498a-af91-ddb44facc578 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.695839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 956/1897] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:24 2022] PUT /identity/v3/projects/b85465d60267462e98351b6154d8e1a7/users/03bf69013f46445ca3d81482dbb7e5e5/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:24.700427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.700720 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.701002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.758758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] MFA Rules not processed for user `03bf69013f46445ca3d81482dbb7e5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:24.760341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:24.799957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.800696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.800981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.801265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.802224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:24.803541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.805241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.824034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 942/1898] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:24.832260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.851772 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bc068-f1f9-4bc3-ae35-338688d8de48 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:24.857576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:24.858519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:24.858870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:24.859536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:24.861116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:24.863859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:24.885344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 957/1899] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:24.890671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.902376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 943/1900] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/projects => generated 389 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:24.904227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:24.915734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ef7ad-9a14-4e9c-8fea-0a6ace42a1a7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.016341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.016958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3eaa2-316e-4586-b245-073d1cf595de admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.017227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.017570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.017869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.018606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.020549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.022910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.025602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.029552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.029928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.030910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.033179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.064366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 958/1901] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:25.069848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.075381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 944/1902] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:24 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:25.079634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.097440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c539eff-acc4-4b60-9c2f-9b14a71c4011 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.113813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:25.161779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:25.162304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d42511-03f6-4fa5-ab51-c7c65f63c9c0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.169232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.170068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.170442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.170840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.171832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.175077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.194067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.194922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.195761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.195761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.196814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.199821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.202292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 959/1903] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:25.206274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.214396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 945/1904] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:25.221412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.222105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93aee1-a7f9-4111-b048-216f5d59c68a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.232279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264c359-889b-49f0-b7ea-1d9a3f6de0f3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.233348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.234439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.234906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.235361 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.236267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.240089 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.240655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.241602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.241912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.242205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.243097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.245443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.255738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 960/1905] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:25.257103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.258160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 946/1906] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:25.262273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.267626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d980-0701-4cc9-97c5-589d33af7a75 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.282819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.283938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.283938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.283938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.285028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.286317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a8cc0-82ad-4adc-9e54-3711f51db54e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.287527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.293058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.294632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.295257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.295739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] PATH_INFO: `/v3/projects/673495da901a4bb78c83f02f8ea3afe9/users/262a2751de0b47459c3a3b53eace6716/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.309261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 961/1907] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:25.310213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.312344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.321350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673495da901a4bb78c83f02f8ea3afe9, user_id=262a2751de0b47459c3a3b53eace6716, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.322351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.338775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b56e8b-04e8-4471-aad0-4c4c1eb6f609 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.347315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 947/1908] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:25 2022] PUT /identity/v3/projects/673495da901a4bb78c83f02f8ea3afe9/users/262a2751de0b47459c3a3b53eace6716/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:25.354505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.412121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.413252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.413902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.414359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] PATH_INFO: `/v3/projects/5e499f12cd36471bbbf9879a5e5af187` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.415635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e499f12cd36471bbbf9879a5e5af187)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.417598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.425177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084c6aa-47c9-4f8e-adaf-504713196a66 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.431657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.432493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.432775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.433060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] PATH_INFO: `/v3/projects/ae08bd15e10e4f88a0062f7f2e306ff4/users/1bccce99cb2e4267b1baa151d877c484/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.452484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae08bd15e10e4f88a0062f7f2e306ff4, user_id=1bccce99cb2e4267b1baa151d877c484, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.455761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6cf1c-81e7-4509-8ed8-31eda7aa87b6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.476280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 948/1909] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:25 2022] PUT /identity/v3/projects/ae08bd15e10e4f88a0062f7f2e306ff4/users/1bccce99cb2e4267b1baa151d877c484/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:25.477630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.477976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.478200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.487460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e499f12cd36471bbbf9879a5e5af187', 'request_id': 'req-6dd22701-f212-493b-849d-1f10ab1d2a77'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.497299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e499f12cd36471bbbf9879a5e5af187', 'request_id': 'req-6dd22701-f212-493b-849d-1f10ab1d2a77'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.505927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] MFA Rules not processed for user `262a2751de0b47459c3a3b53eace6716`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:25.507631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.507743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e499f12cd36471bbbf9879a5e5af187', 'request_id': 'req-6dd22701-f212-493b-849d-1f10ab1d2a77'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.521321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] The token cache is being invalidate because project 5e499f12cd36471bbbf9879a5e5af187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:25.522003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.522881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6dd22701-f212-493b-849d-1f10ab1d2a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.524610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 962/1910] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:25 2022] DELETE /identity/v3/projects/5e499f12cd36471bbbf9879a5e5af187 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:25.527891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.528180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.529104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.562566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] MFA Rules not processed for user `1bccce99cb2e4267b1baa151d877c484`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:25.563772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.572776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:25.582428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 949/1911] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:25.583273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.584648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.613770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:25.653770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 963/1912] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:25.654809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.656094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.671605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aedd52-6504-47fe-81f1-0147927541e5 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.673401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d56f2-c829-4487-af3c-2af67196d13e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.678623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.679142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.679288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.679563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.679841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.680149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.680436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.680548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.680707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] PATH_INFO: `/v3/users/ea1ae81cc1dd4b3394396c2830741e9a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.681970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1ae81cc1dd4b3394396c2830741e9a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.682461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.684152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.719265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 964/1913] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:25.720158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.722940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.743962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1ae81cc1dd4b3394396c2830741e9a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.744857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1ae81cc1dd4b3394396c2830741e9a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.764426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1ae81cc1dd4b3394396c2830741e9a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.777713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1ae81cc1dd4b3394396c2830741e9a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.779594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1ae81cc1dd4b3394396c2830741e9a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:25.798846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 950/1914] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:25 2022] DELETE /identity/v3/users/ea1ae81cc1dd4b3394396c2830741e9a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:25.800072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.801989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:25.819565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526e788-8921-4374-9a23-f3d6dea85c19 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.908698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd50-4c83-421e-902c-3f59e9c0a36f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.915419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.916290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.916610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.916889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.917589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.920281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.956924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 951/1915] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/users => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:25.957096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.957096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.957096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.957096 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.957956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:25.958059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:25.965828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:25.980608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25787-1030-4181-a033-897e4decc870 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:25.994878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:25.996091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:25.997167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:25.998519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:25.999278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.004077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.023727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 965/1916] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:25 2022] POST /identity/v3/projects => generated 379 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:26.027740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.030779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.031312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 952/1917] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:26.038479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.053434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b7aed-0a81-4e4f-a3dc-5b9e8b33051c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.066766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.099803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.170100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.171080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.171431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.172001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] PATH_INFO: `/v3/projects/5d11f7210b19419aaf3af4fb6fde3182/users/0b1e97be90f64a319532083dfde772fc/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.172386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85abc2-89af-439e-85ac-1e9c853c0513 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.181624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.183563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.184321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.184651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.185358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.190894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.233358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d11f7210b19419aaf3af4fb6fde3182, user_id=0b1e97be90f64a319532083dfde772fc, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.238594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.243469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 966/1918] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:26 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:26.249194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.263027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 953/1919] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:26 2022] PUT /identity/v3/projects/5d11f7210b19419aaf3af4fb6fde3182/users/0b1e97be90f64a319532083dfde772fc/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:26.270625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.273457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9851df-6466-4a5e-aee3-a9d852a6e003 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.386299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252d883-fb12-48c4-b691-4c9ba6dcd7bd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.391637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.392501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.394287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.394814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.395447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.396668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.399584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.402144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.402611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.402954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.403902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.405542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.417748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 954/1920] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:26.418804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.421303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.425778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 967/1921] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:26.426856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.441025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c16e6-7de6-41df-b03d-9115c4c5cec7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.450453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5de73-a149-4fa2-a037-5954650f1ec9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.454777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.456674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.456967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.457245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] PATH_INFO: `/v3/projects/5d11f7210b19419aaf3af4fb6fde3182/users/0b1e97be90f64a319532083dfde772fc/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.461278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d11f7210b19419aaf3af4fb6fde3182, user_id=0b1e97be90f64a319532083dfde772fc, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.463208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.464381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.464544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.464854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.465461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] PATH_INFO: `/v3/projects/54206c45a873429a971e1908f734026f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.467104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=54206c45a873429a971e1908f734026f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.469351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.489183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 968/1922] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:26 2022] PUT /identity/v3/projects/5d11f7210b19419aaf3af4fb6fde3182/users/0b1e97be90f64a319532083dfde772fc/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:26.489183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.545458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54206c45a873429a971e1908f734026f', 'request_id': 'req-00a25072-b2d9-4fb5-894a-c9349f3555ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.553540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54206c45a873429a971e1908f734026f', 'request_id': 'req-00a25072-b2d9-4fb5-894a-c9349f3555ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.561241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54206c45a873429a971e1908f734026f', 'request_id': 'req-00a25072-b2d9-4fb5-894a-c9349f3555ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.585115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] The token cache is being invalidate because project 54206c45a873429a971e1908f734026f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:26.585528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.586810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00a25072-b2d9-4fb5-894a-c9349f3555ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.590374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 955/1923] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:26 2022] DELETE /identity/v3/projects/54206c45a873429a971e1908f734026f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:26.593874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a898-c903-41a3-9705-0f529fdff786 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.593874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a898-c903-41a3-9705-0f529fdff786 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.593874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6a898-c903-41a3-9705-0f529fdff786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.607013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584cc77-80d7-48e6-8b66-47c35c9bff77 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.627103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-23d6a898-c903-41a3-9705-0f529fdff786 None None] MFA Rules not processed for user `0b1e97be90f64a319532083dfde772fc`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:26.628960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-23d6a898-c903-41a3-9705-0f529fdff786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.633135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.659862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.684445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.685359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.685639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.685915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] PATH_INFO: `/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99/users/dc1c3feb179f4463a3c9c97a0b0c3e3a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.700907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14c18e982f847dfb7ff0c2d0ddfcb99, user_id=dc1c3feb179f4463a3c9c97a0b0c3e3a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.703345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 956/1924] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:26.703637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.704798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.705092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.705394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.718000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 969/1925] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:26 2022] PUT /identity/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99/users/dc1c3feb179f4463a3c9c97a0b0c3e3a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:26.718925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.720590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.727422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:26.728591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.786426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea2174-7fb9-4185-8875-2f997a357c25 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.790128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 957/1926] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:26.791273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.792146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.792425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.792694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] PATH_INFO: `/v3/users/af3ab8b1dced46f2a164ea3f35de80bf` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.793900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3ab8b1dced46f2a164ea3f35de80bf)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.795211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.795904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.796676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.843544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3ab8b1dced46f2a164ea3f35de80bf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.851474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3ab8b1dced46f2a164ea3f35de80bf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.852261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3ab8b1dced46f2a164ea3f35de80bf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.855783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148c386-7976-4d61-92d3-03c6e071fabf None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.859810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.860449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.860735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.861116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.865226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3ab8b1dced46f2a164ea3f35de80bf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.865932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3ab8b1dced46f2a164ea3f35de80bf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:26.878771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 970/1927] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:26 2022] DELETE /identity/v3/users/af3ab8b1dced46f2a164ea3f35de80bf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:26.879564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.880903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.915455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.925813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.927243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:26.927359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb8735-7ee2-4f4d-ad87-087d7486cb46 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.937233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 958/1928] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:26.938970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131cdd-d32d-4308-a503-a1b792d4bc79 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.939274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131cdd-d32d-4308-a503-a1b792d4bc79 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.939537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131cdd-d32d-4308-a503-a1b792d4bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.952278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c201a5c-e29a-48d2-bb12-be4f9f3c3d8d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:26.959084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8f131cdd-d32d-4308-a503-a1b792d4bc79 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:26.959670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8f131cdd-d32d-4308-a503-a1b792d4bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:26.963923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:26.964808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.965191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.965677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.966772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:26.969005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:26.969163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 959/1929] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:26.974557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:26.974856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:26.975136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:26.988601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 971/1930] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:26.988601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:26.994051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:26.995444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.003121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a127f6-d79f-48d3-8e68-47295f58228b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.004425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 960/1931] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:27.007574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.008504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.009276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.009566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.009844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.010603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:27.010917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.012517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.024662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4432085-2c3e-4ab4-bb74-4446f81fc9db None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.031237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 972/1932] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:27.031237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.033199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.042062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.043248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.043651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.043948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] PATH_INFO: `/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99/users/dc1c3feb179f4463a3c9c97a0b0c3e3a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.047057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14c18e982f847dfb7ff0c2d0ddfcb99, user_id=dc1c3feb179f4463a3c9c97a0b0c3e3a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.050584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.062364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 961/1933] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:27 2022] PUT /identity/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99/users/dc1c3feb179f4463a3c9c97a0b0c3e3a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:27.071023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.102424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56281c3a-ec9e-46ee-8498-f0f2155324df admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.155390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.155636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e03f04-6430-4269-95ce-2bdd06d69f29 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.156196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.156488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.156835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.160982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.161639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.161925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.162202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.162921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.166111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.167507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.168942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96da966-279e-4de9-8f1f-fa404c6dce9b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.180085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 973/1934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:27.188092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.188437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.188903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.203867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 962/1935] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:27 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:27.209285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.212026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.262307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] MFA Rules not processed for user `dc1c3feb179f4463a3c9c97a0b0c3e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:27.265494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.308826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5f3d9-a460-48e2-86d5-888eeeb1fe13 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.315739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.316875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.317260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.317667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] PATH_INFO: `/v3/users/262a2751de0b47459c3a3b53eace6716` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.319051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=262a2751de0b47459c3a3b53eace6716)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.322567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.425932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262a2751de0b47459c3a3b53eace6716'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:27.426313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262a2751de0b47459c3a3b53eace6716'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:27.432321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:27.435316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 974/1936] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:27.436402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.448150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262a2751de0b47459c3a3b53eace6716'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:27.462650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262a2751de0b47459c3a3b53eace6716'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:27.463515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262a2751de0b47459c3a3b53eace6716'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:27.471532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d46cab-9a58-4ec7-90dc-1a8291ccb01e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.477413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.478099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.478414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.478701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.479412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.482453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c3479-8f59-420a-9574-d334f3610241 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.483456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 963/1937] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:27 2022] DELETE /identity/v3/users/262a2751de0b47459c3a3b53eace6716 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:27.485023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.488458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.497255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:27.519665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 975/1938] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:27 2022] POST /identity/v3/users => generated 279 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:27.521444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.521742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.522178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.550926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:27.552472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.577767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d020d-91aa-4e54-8b96-c6ab7d4edfdb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.584283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.585784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.586838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.587348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.589560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.662579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 976/1939] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:27.667117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.668755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.677302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.678984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.683831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534cc7-fc02-40bb-9fb4-214d5126ee28 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.691337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 964/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:27.695681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.697359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.697645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.697921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.698971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.701411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.718514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.727556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 977/1941] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:27.734086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b28fc-fff2-452b-8847-5e9e411b7375 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.741082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.742051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.742468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.742863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.743790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.745647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.747435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.754163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.775127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 965/1942] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:27.776384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.781532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.809748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9525-b630-4b83-9c5b-fcb1db6b297c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.813722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.814424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.815267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.815547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.817567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.834563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd4f7-0f59-41a2-92a6-bd9f65c00bfe admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.838936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.839988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.840589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.841460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.854676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.856175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.866831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 978/1943] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:27.867717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.869130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:27.885854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.887505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.899832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3db46-6308-4bc6-9f0e-516eb56ffecb service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.900540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 966/1944] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:27.901607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:27.915253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.916480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.916881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.917238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] PATH_INFO: `/v3/projects/17cef16b0169473ca88243c07c811cc0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.922210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=17cef16b0169473ca88243c07c811cc0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.922550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c876f1-3cb0-42b7-93e5-89d36013df1d service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:27.927421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:27.943987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:27.944899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:27.945179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:27.945454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] PATH_INFO: `/v3/projects/9d599fc15c1c4b1e9419484a43c5298a/users/4ab4e3ebbe44446cb34dd850b0dcf2bc/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:27.949849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:27.973127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d599fc15c1c4b1e9419484a43c5298a, user_id=4ab4e3ebbe44446cb34dd850b0dcf2bc, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:27.974203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994dd02-9cea-4411-8fad-6eef546363f8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.003215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 967/1945] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:27 2022] PUT /identity/v3/projects/9d599fc15c1c4b1e9419484a43c5298a/users/4ab4e3ebbe44446cb34dd850b0dcf2bc/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.015506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.016058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.016521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.056740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] MFA Rules not processed for user `4ab4e3ebbe44446cb34dd850b0dcf2bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:28.058193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17cef16b0169473ca88243c07c811cc0', 'request_id': 'req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.059271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.066689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17cef16b0169473ca88243c07c811cc0', 'request_id': 'req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.080279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17cef16b0169473ca88243c07c811cc0', 'request_id': 'req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.108811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] The token cache is being invalidate because project 17cef16b0169473ca88243c07c811cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:28.109146 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.109841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.115266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 979/1946] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:27 2022] DELETE /identity/v3/projects/17cef16b0169473ca88243c07c811cc0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.116304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.117689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.128946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:28.160813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 968/1947] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:28.165296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.167537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.200979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62d5b8-0fb6-4274-b6b9-a91135cce6ca admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.208347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.211060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.211342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.211621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] PATH_INFO: `/v3/users/17c9b0923800467380122f3564017991` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.213574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c9b0923800467380122f3564017991)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.215544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.279636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b3100-70d0-441a-97b8-38f9809d7aed None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.285587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c9b0923800467380122f3564017991'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.287607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.288347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.288560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.288843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.289680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.291650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.325709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c9b0923800467380122f3564017991'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.327850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c9b0923800467380122f3564017991'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.337761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 969/1948] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:28 2022] POST /identity/v3/projects => generated 355 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:28.339349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.341820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.354317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c9b0923800467380122f3564017991'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.356302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c9b0923800467380122f3564017991'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.372317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 980/1949] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:28 2022] DELETE /identity/v3/users/17c9b0923800467380122f3564017991 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.375241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.378424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.447400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:28.453658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca709b6-8138-4246-9195-5fc2ce547d6b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.495162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc831b-fe94-47bf-ae9a-cf98e13e3aba admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.501282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.502171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.502828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.503259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.504047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.506740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.535899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 981/1950] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:28 2022] POST /identity/v3/users => generated 284 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:28.539907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.553615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c578a5-d75f-435e-9ec8-8b2a9a34806d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.558717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.559614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.559913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.560907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] PATH_INFO: `/v3/projects/673495da901a4bb78c83f02f8ea3afe9` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.562238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673495da901a4bb78c83f02f8ea3afe9)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.564191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.566650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.568177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.568461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.568738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.572657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.575413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.604102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 982/1951] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:28.608962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.617199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:28.622789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede20436-0374-4584-8274-e7b6f8ddd38c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.635163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.636120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.636452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.637033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] PATH_INFO: `/v3/projects/90d0c8429a754800bbf952e05ba4b393/users/9c0ab1a89a264113985bb8757fa40677/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.657122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673495da901a4bb78c83f02f8ea3afe9', 'request_id': 'req-ad8790e7-ed53-463f-8a96-61a4da6f4377'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.657823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d0c8429a754800bbf952e05ba4b393, user_id=9c0ab1a89a264113985bb8757fa40677, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.663792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.669524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673495da901a4bb78c83f02f8ea3afe9', 'request_id': 'req-ad8790e7-ed53-463f-8a96-61a4da6f4377'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.679001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673495da901a4bb78c83f02f8ea3afe9', 'request_id': 'req-ad8790e7-ed53-463f-8a96-61a4da6f4377'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.680081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 983/1952] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:28 2022] PUT /identity/v3/projects/90d0c8429a754800bbf952e05ba4b393/users/9c0ab1a89a264113985bb8757fa40677/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.684143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.698998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] The token cache is being invalidate because project 673495da901a4bb78c83f02f8ea3afe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:28.699616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.700478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.702894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 970/1953] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:28 2022] DELETE /identity/v3/projects/673495da901a4bb78c83f02f8ea3afe9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.707552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.709242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.770158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226a533-fd21-4116-a978-b0c7c5a52ff8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.777185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.778204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.778576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.778931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.780103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.781460 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.794714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 984/1954] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:28.799043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.800878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.815551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862590d1-6cfd-43fe-8b9a-810a59d200db admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.815722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8790e7-ed53-463f-8a96-61a4da6f4377 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.820288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.821220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.821501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.821901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] PATH_INFO: `/v3/projects/90d0c8429a754800bbf952e05ba4b393/users/9c0ab1a89a264113985bb8757fa40677/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.823053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.824428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.824550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d0c8429a754800bbf952e05ba4b393, user_id=9c0ab1a89a264113985bb8757fa40677, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.824776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.825132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] PATH_INFO: `/v3/users/0b1e97be90f64a319532083dfde772fc` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.826526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1e97be90f64a319532083dfde772fc)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.827342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.836126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.841795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 985/1955] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:28 2022] PUT /identity/v3/projects/90d0c8429a754800bbf952e05ba4b393/users/9c0ab1a89a264113985bb8757fa40677/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.850954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:28.854137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:28.891559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:28.901784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1e97be90f64a319532083dfde772fc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.902607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1e97be90f64a319532083dfde772fc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.923588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1e97be90f64a319532083dfde772fc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.933285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38a60a-4f5a-4cdb-af20-17d40e664adb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:28.935066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1e97be90f64a319532083dfde772fc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.935832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e39d1fe-caa0-451e-940d-cfcce5ab9d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1e97be90f64a319532083dfde772fc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:28.938464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:28.939543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.940721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.941005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] PATH_INFO: `/v3/projects/ce12e97571f94917a5d4e866496aacb3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:28.942373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce12e97571f94917a5d4e866496aacb3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:28.944787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:28.953203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 971/1956] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:28 2022] DELETE /identity/v3/users/0b1e97be90f64a319532083dfde772fc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:28.962607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:28.962607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:28.962607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.004950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] MFA Rules not processed for user `9c0ab1a89a264113985bb8757fa40677`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.007482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.024344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce12e97571f94917a5d4e866496aacb3', 'request_id': 'req-b3213d85-9455-4946-bf18-f116f80eb174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.034490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce12e97571f94917a5d4e866496aacb3', 'request_id': 'req-b3213d85-9455-4946-bf18-f116f80eb174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.046540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce12e97571f94917a5d4e866496aacb3', 'request_id': 'req-b3213d85-9455-4946-bf18-f116f80eb174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.048934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:29.067892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] The token cache is being invalidate because project ce12e97571f94917a5d4e866496aacb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:29.068524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.070624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b3213d85-9455-4946-bf18-f116f80eb174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.072569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 986/1957] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:28 2022] DELETE /identity/v3/projects/ce12e97571f94917a5d4e866496aacb3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:29.078663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe7c1-c56d-46df-b538-2566769463cf None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.079607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe7c1-c56d-46df-b538-2566769463cf None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.079904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fe7c1-c56d-46df-b538-2566769463cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.105099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b64fe7c1-c56d-46df-b538-2566769463cf None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.106364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b64fe7c1-c56d-46df-b538-2566769463cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.140134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 972/1958] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:29.149770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.151699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.175404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 987/1959] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:29.183284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.183842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.184198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.203960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.205857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.214367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 988/1960] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:29.215790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949fcfd-872b-4c16-9dcb-8300b10213f4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.218220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.220061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.220870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.221227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.221580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.229605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64876f73-02b7-4045-a1ea-b07f21f3f108 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.232411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.233921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.234716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.235434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.235784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.236123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.236942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:29.237405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.239209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.244781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 973/1961] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:29.254861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 989/1962] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:53:29.259671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.274422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.276230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.277513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b770-22ad-4741-9f61-08073723e4c3 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.286370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.287148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.290967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.290967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.290967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.290967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.300565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8f1b4-de70-44b6-af5c-7429f608fa9a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.305793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.306714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.307465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.307465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] PATH_INFO: `/v3/projects/5d11f7210b19419aaf3af4fb6fde3182` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.308522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d11f7210b19419aaf3af4fb6fde3182)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.310631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.331762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 974/1963] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:29.333605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.357157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:29.369828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d11f7210b19419aaf3af4fb6fde3182', 'request_id': 'req-05001b4c-d473-4cf8-ab2e-2f867a806daf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.376055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d11f7210b19419aaf3af4fb6fde3182', 'request_id': 'req-05001b4c-d473-4cf8-ab2e-2f867a806daf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.387948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d11f7210b19419aaf3af4fb6fde3182', 'request_id': 'req-05001b4c-d473-4cf8-ab2e-2f867a806daf'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.396986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294db3b-ab60-4f4e-acf0-8b30faaecfc8 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.403342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] The token cache is being invalidate because project 5d11f7210b19419aaf3af4fb6fde3182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:29.403908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.404952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:29.406828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 990/1964] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:29 2022] DELETE /identity/v3/projects/5d11f7210b19419aaf3af4fb6fde3182 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:29.407590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.408920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.458607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.458607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.458607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.458607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.458894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.483489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:29.489630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001b4c-d473-4cf8-ab2e-2f867a806daf admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.501523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.502691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.502958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.503237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.503920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.506062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c642f-ee3e-45a0-a48f-6122150e3dde admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.530043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 991/1965] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:29.530043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc662-805b-4d56-b053-46b3cca393fd None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.530043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc662-805b-4d56-b053-46b3cca393fd None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.530043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafc662-805b-4d56-b053-46b3cca393fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.536325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.538129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.547202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9cafc662-805b-4d56-b053-46b3cca393fd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.548402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9cafc662-805b-4d56-b053-46b3cca393fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.549456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 975/1966] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:29.551893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.557703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 992/1967] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:29.565562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903405-f1e2-4e69-bf8f-9e44f2ac210f service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.567494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.567845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.568195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.571046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.572164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.572521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.572805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.574561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.576513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.593838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.593941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 976/1968] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:29.594028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.594475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.605031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 993/1969] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:29.605563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307db80d-af2a-47d4-a5a7-0088cd465752 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.606046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.610948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.612244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.612957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.613384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.616502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.618382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.623113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4d358-3cff-4829-b600-7389e573b13b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.630188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.631158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.631741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.632007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.632876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:29.633551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.634574 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 977/1970] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:29.635328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.635514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.648305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537550c6-b028-421c-9b7a-61a6d2ccf994 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.651340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 994/1971] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:29.658116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.658243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.659190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.659576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.660069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] PATH_INFO: `/v3/projects/197b70c1fe7742ab9b23099858ffd558/users/c9c1e4b958e4492eb96de9cc5d8d93a6/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.682692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=197b70c1fe7742ab9b23099858ffd558, user_id=c9c1e4b958e4492eb96de9cc5d8d93a6, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.685891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f4778-fd22-4037-a634-58b664b6c6af admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.688543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cd185-9217-4b84-a80a-ad27480e49af admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:29.699352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:29.702973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.704585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.705104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.709363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:29.713876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:29.722420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 978/1972] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:29 2022] PUT /identity/v3/projects/197b70c1fe7742ab9b23099858ffd558/users/c9c1e4b958e4492eb96de9cc5d8d93a6/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:29.727476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:29.727764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:29.728635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:29.778405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] MFA Rules not processed for user `c9c1e4b958e4492eb96de9cc5d8d93a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:29.784277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:29.791987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 995/1973] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:29.801836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:29.886559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:29.995885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c10542-160b-4868-a3b6-a610734ab51a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.015055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.015739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.016029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.016284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.016978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.019173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.025364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 979/1974] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:30.026851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.073541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 996/1975] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:29 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:30.074415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.098499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fafc8-f8ce-4a88-b4b7-9bbf4f0cf7ac admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.113637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.114424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.114794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.115271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.116043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.117738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.149410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 997/1976] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:30.150369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.183317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66babf-9903-4666-ad23-3f42eabdd86d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.195654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.196439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.196669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.196949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.197734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.202647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.230542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 998/1977] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:30.235131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.261395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef145e1-d529-4086-9a63-778aff5a8adc admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.272166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.272166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.274590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.275174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] PATH_INFO: `/v3/projects/a00ba85fadb54b82b8f08c17fc1810b9/users/822c13c8722c40628e2e972b9e5b4283/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.280113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753ebfd-0e1b-4e1a-b2d5-4bd865a41c47 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.292276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.293356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.293825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.294283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.295544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.312242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.347635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00ba85fadb54b82b8f08c17fc1810b9, user_id=822c13c8722c40628e2e972b9e5b4283, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.355627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ac87b-5a3c-4999-9799-5af288964118 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.384012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 980/1978] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/projects => generated 365 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:30.398040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.416700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 999/1979] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:30 2022] PUT /identity/v3/projects/a00ba85fadb54b82b8f08c17fc1810b9/users/822c13c8722c40628e2e972b9e5b4283/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:30.422936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.423455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.424575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.486539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] MFA Rules not processed for user `822c13c8722c40628e2e972b9e5b4283`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:30.488155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:30.501804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ba5db-be9c-47b5-aeae-d44c305a5b72 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.537452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:30.576863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.577543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.578089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.578089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.578834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.581546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.595471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1000/1980] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:30.600091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.604254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:30.622220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 981/1981] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:30.626434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.628480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d3afe-22c0-40f9-8624-c653ca44b133 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.634586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.635461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.636014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.636507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.637407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.637592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4625d3-d2b5-4395-adc7-e2e412375449 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.639876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.643539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.644270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.644814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.645168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.645964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.647658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.658662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 982/1982] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:30.663351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.676001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1001/1983] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:30.676161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca4fe3-d40a-4b66-ba53-f4215ebff6d5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.676913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.741578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.742135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.742396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.742811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] PATH_INFO: `/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383/users/63353655491941f09a0654844409790a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.744610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73dd71-a293-4b84-8376-46beaa96bd6d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.753023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.753716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.753994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.754268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.754984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.757347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.758353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19d27a1c42b4ecb87f6c6338b44a383, user_id=63353655491941f09a0654844409790a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.761213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.776937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 983/1984] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:30 2022] PUT /identity/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383/users/63353655491941f09a0654844409790a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:30.778715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1002/1985] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:30.781168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.782352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.791923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3513d-b4c8-46a6-b8ee-99564d2dc2ad admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.846994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68c6e-0749-4831-8203-c9e06e6d2c7c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.852881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.853159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.853547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.853811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.853944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.854125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.854222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.854412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.854875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.855106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.856475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.856701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.875324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 984/1986] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:30.876059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1003/1987] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:30.876856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.878446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.887806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648c9a4-d7bb-47f6-b1ec-5a9235c1c66c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.891340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f6336-4551-464d-af80-9bb62a4bbc3d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:30.894166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.895359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.895815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.896104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] PATH_INFO: `/v3/projects/7a0ffdda57a64e71b22980c48a492bcb/users/c2b85c53b9444d5e99149de6b02cba00/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.898901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:30.899962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.900247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.900687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] PATH_INFO: `/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383/users/63353655491941f09a0654844409790a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.902905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19d27a1c42b4ecb87f6c6338b44a383, user_id=63353655491941f09a0654844409790a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.905918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2dd98-aea8-4d55-92cf-a0a372c3fab2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.915507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0ffdda57a64e71b22980c48a492bcb, user_id=c2b85c53b9444d5e99149de6b02cba00, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:30.918917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:30.919149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 985/1988] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:30 2022] PUT /identity/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383/users/63353655491941f09a0654844409790a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:30.931491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a697cf-7e7b-4a7b-955e-98818f62a130 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:30.932022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a697cf-7e7b-4a7b-955e-98818f62a130 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:30.932139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a697cf-7e7b-4a7b-955e-98818f62a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:30.938385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1004/1989] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:30 2022] PUT /identity/v3/projects/7a0ffdda57a64e71b22980c48a492bcb/users/c2b85c53b9444d5e99149de6b02cba00/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:30.943316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:30.961762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:30.983212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e0a697cf-7e7b-4a7b-955e-98818f62a130 None None] MFA Rules not processed for user `63353655491941f09a0654844409790a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:30.984931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e0a697cf-7e7b-4a7b-955e-98818f62a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:31.049742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a355-f7cb-46bc-be83-a90923f9c28b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.058541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.059234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.059521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.059803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.060540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.063543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.084501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:31.089080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1005/1990] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:31.093178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:31.108739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731a886-a258-4392-8739-89e486cd6a52 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.115615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.117339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.117624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.117982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] PATH_INFO: `/v3/projects/7a0ffdda57a64e71b22980c48a492bcb/users/c2b85c53b9444d5e99149de6b02cba00/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.120247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0ffdda57a64e71b22980c48a492bcb, user_id=c2b85c53b9444d5e99149de6b02cba00, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.123213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.136488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 986/1991] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:31.139003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1006/1992] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:31 2022] PUT /identity/v3/projects/7a0ffdda57a64e71b22980c48a492bcb/users/c2b85c53b9444d5e99149de6b02cba00/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:31.148325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.148694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.149058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.150030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:31.152152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:31.180070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] MFA Rules not processed for user `c2b85c53b9444d5e99149de6b02cba00`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:31.181874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:31.228066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d00f-d307-4527-a5b0-df49061275d7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.232024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.232677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.232963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.233244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.256569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 987/1993] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:31.267402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:31.268751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:31.280600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5521081-6ce7-4d40-a5f3-95dc5934e832 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.284591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.285232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.285518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.285798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.294458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.295936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.306626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1007/1994] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:31.352585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.354023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.357813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:31.364053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 988/1995] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:31.367558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a760d5-8cc9-409c-a539-7f70603923e1 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.371758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.372662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.373012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.373471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.427940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:31.445595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.447278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad8f99-4799-4343-9538-65380d021a2c service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.458051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1008/1996] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:31.835733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:31.837190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:31.902447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d915a-f66a-4014-99f0-b42d53120b54 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:31.907008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:31.907873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.908178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.908457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] PATH_INFO: `/v3/users/03bf69013f46445ca3d81482dbb7e5e5` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:31.909636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bf69013f46445ca3d81482dbb7e5e5)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:31.911588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:31.913243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:31.956589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bf69013f46445ca3d81482dbb7e5e5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:31.957268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bf69013f46445ca3d81482dbb7e5e5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:31.969027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bf69013f46445ca3d81482dbb7e5e5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:31.976166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bf69013f46445ca3d81482dbb7e5e5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:31.976810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bf69013f46445ca3d81482dbb7e5e5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:31.989492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 989/1997] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:31 2022] DELETE /identity/v3/users/03bf69013f46445ca3d81482dbb7e5e5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:31.994254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:31.994577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:31.994887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:32.013749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:32.014992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:32.073579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1009/1998] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:32.083528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:32.084918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:32.143165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7963efa-1a40-4f61-b73f-804bf27e2164 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:32.146835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:32.147479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:32.147770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:32.148038 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:32.157064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:32.159028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:32.168605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 990/1999] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:32.449098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:32.450669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:32.476074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd46ba-bf5f-47f0-b6de-3b053f709983 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:32.492043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:32.493000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:32.493284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:32.493564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] PATH_INFO: `/v3/projects/b85465d60267462e98351b6154d8e1a7` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:32.494964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b85465d60267462e98351b6154d8e1a7)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:32.497046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:32.551272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85465d60267462e98351b6154d8e1a7', 'request_id': 'req-476cc6b7-36fc-4431-a647-7289c02ede1a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.557006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85465d60267462e98351b6154d8e1a7', 'request_id': 'req-476cc6b7-36fc-4431-a647-7289c02ede1a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.565547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85465d60267462e98351b6154d8e1a7', 'request_id': 'req-476cc6b7-36fc-4431-a647-7289c02ede1a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.577133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] The token cache is being invalidate because project b85465d60267462e98351b6154d8e1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:32.577422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.578067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.579660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1010/2000] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:32 2022] DELETE /identity/v3/projects/b85465d60267462e98351b6154d8e1a7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:32.583831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:32.593763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7d3b8-c5e2-464c-b758-64a16fd1ad10 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:32.651885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:32.652679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:32.652960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:32.653245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] PATH_INFO: `/v3/users/1bccce99cb2e4267b1baa151d877c484` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:32.654479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bccce99cb2e4267b1baa151d877c484)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:32.656431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:32.679575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:32.706192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bccce99cb2e4267b1baa151d877c484'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.706913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bccce99cb2e4267b1baa151d877c484'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.718927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bccce99cb2e4267b1baa151d877c484'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.726436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bccce99cb2e4267b1baa151d877c484'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.727056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bccce99cb2e4267b1baa151d877c484'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:32.740553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 991/2001] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:32 2022] DELETE /identity/v3/users/1bccce99cb2e4267b1baa151d877c484 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:33.116956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:33.118662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:33.134787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc6b7-36fc-4431-a647-7289c02ede1a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:33.200452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:33.201580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:33.201773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:33.202051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] PATH_INFO: `/v3/projects/ae08bd15e10e4f88a0062f7f2e306ff4` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:33.203387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae08bd15e10e4f88a0062f7f2e306ff4)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:33.205738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:33.213960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:33.275534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae08bd15e10e4f88a0062f7f2e306ff4', 'request_id': 'req-36fba449-0f6e-43e6-8d0a-f385b2dd9566'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.281652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae08bd15e10e4f88a0062f7f2e306ff4', 'request_id': 'req-36fba449-0f6e-43e6-8d0a-f385b2dd9566'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.291727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae08bd15e10e4f88a0062f7f2e306ff4', 'request_id': 'req-36fba449-0f6e-43e6-8d0a-f385b2dd9566'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.304106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] The token cache is being invalidate because project ae08bd15e10e4f88a0062f7f2e306ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:33.304616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.305344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.307347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1011/2002] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:33 2022] DELETE /identity/v3/projects/ae08bd15e10e4f88a0062f7f2e306ff4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:33.311210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:33.312831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:33.327410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb9616-3595-4f8e-a456-1016ff60e8a0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:33.384428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:33.385338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:33.385627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:33.385905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] PATH_INFO: `/v3/users/dc1c3feb179f4463a3c9c97a0b0c3e3a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:33.387118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1c3feb179f4463a3c9c97a0b0c3e3a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:33.389039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:33.437305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1c3feb179f4463a3c9c97a0b0c3e3a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.437875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1c3feb179f4463a3c9c97a0b0c3e3a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.451224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1c3feb179f4463a3c9c97a0b0c3e3a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.459856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1c3feb179f4463a3c9c97a0b0c3e3a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.460476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bea51980-42fb-4038-a28a-85514c29595b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1c3feb179f4463a3c9c97a0b0c3e3a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.473287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 992/2003] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:33 2022] DELETE /identity/v3/users/dc1c3feb179f4463a3c9c97a0b0c3e3a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:33.758568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:33.760092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:33.774548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fba449-0f6e-43e6-8d0a-f385b2dd9566 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:33.843000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:33.844032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:33.844371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:33.844666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] PATH_INFO: `/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:33.846601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14c18e982f847dfb7ff0c2d0ddfcb99)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:33.848894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:33.897289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14c18e982f847dfb7ff0c2d0ddfcb99', 'request_id': 'req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.902612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14c18e982f847dfb7ff0c2d0ddfcb99', 'request_id': 'req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.909352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14c18e982f847dfb7ff0c2d0ddfcb99', 'request_id': 'req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.920201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] The token cache is being invalidate because project d14c18e982f847dfb7ff0c2d0ddfcb99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:33.920486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.921106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-efa00d8e-c294-4a65-bbb1-9cde6dc6afea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:33.922688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1012/2004] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:33 2022] DELETE /identity/v3/projects/d14c18e982f847dfb7ff0c2d0ddfcb99 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:33.930552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:33.930846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:33.931121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:33.949695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:33.950867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:33.963823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:34.012556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 993/2005] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:34.020733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.021030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.021432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.040442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:34.041661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.050264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1013/2006] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:34.054642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.064074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4cad4-d61c-4b8c-bfdf-54695505945e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.069577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.070250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.070556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.070830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.071554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:34.071863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.073720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.085885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 994/2007] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:34.090399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.099835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c8349-e6f5-46a3-9e2b-b1b7c0d855e8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.104670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.105326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.105607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.105882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.106631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.108590 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.128944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:34.136922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1014/2008] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:34 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:34.140857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.150863 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d741e1-b228-4f00-a37d-b1840eb8575c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.207898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.208494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.208778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.209055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.209734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.211684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.230170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 995/2009] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:34 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:34.234469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.245469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7c946-e345-4d59-a57a-3b7de1d19759 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.250078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.250764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.251051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.251330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.252684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.254285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.264533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1015/2010] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:34.268357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.278017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9166682-9585-4a1e-a2aa-2a97418e6480 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.282659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.283332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.283620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.283900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.284619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.286202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a08cb-6e86-4fde-9181-4ed58a37f13b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.295941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 996/2011] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:34.300249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.309919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf2f52-191d-4cc0-86f4-86c9fa8d6d17 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.315190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.316055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.316339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.316615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] PATH_INFO: `/v3/projects/bba7d59ae92b4eaebe83a69815471696/users/8578149a810f4841b3cacaad46f12c6e/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.331711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba7d59ae92b4eaebe83a69815471696, user_id=8578149a810f4841b3cacaad46f12c6e, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.334399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.347682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1016/2012] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:34 2022] PUT /identity/v3/projects/bba7d59ae92b4eaebe83a69815471696/users/8578149a810f4841b3cacaad46f12c6e/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:34.358755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.359386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.359684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.389183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] MFA Rules not processed for user `8578149a810f4841b3cacaad46f12c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:34.390524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.459608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 997/2013] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:34.471416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.473282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.533731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce399a0-7399-4033-add6-f494900d164c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.537635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.538423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.538710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.538985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.547510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.549618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee533b3-5fb8-4502-8126-b9c86625f3dc service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.559102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1017/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:34.579275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.608534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:34.676750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ec23c-0ec5-4994-b342-f3755211b1bf None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.683658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.684686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.685147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.685632 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] PATH_INFO: `/v3/users/8578149a810f4841b3cacaad46f12c6e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.687601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8578149a810f4841b3cacaad46f12c6e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.690689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:34.750262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8578149a810f4841b3cacaad46f12c6e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:34.751307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8578149a810f4841b3cacaad46f12c6e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:34.767096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8578149a810f4841b3cacaad46f12c6e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:34.777135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8578149a810f4841b3cacaad46f12c6e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:34.777876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8578149a810f4841b3cacaad46f12c6e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:34.793661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 998/2015] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:34 2022] DELETE /identity/v3/users/8578149a810f4841b3cacaad46f12c6e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:34.798570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.799094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.799187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.816902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:34.818118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.911676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1018/2016] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:34.917042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.918803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.954589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:34.956888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:34.971538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e849f4c-75d9-4202-aa62-9f3a064d6d96 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:34.978242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:34.978878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:34.979166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:34.979445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] PATH_INFO: `/v3/users/c9c1e4b958e4492eb96de9cc5d8d93a6` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:34.980769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c1e4b958e4492eb96de9cc5d8d93a6)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:34.982761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.002084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f25892-52bf-4cbc-aa53-890ba12284ff admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.005438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.006992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.007821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.008239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.020358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.021812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e5ce0-d03a-4596-8ee7-9cfe2eecadcb service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.036406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 999/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:35.045342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c1e4b958e4492eb96de9cc5d8d93a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.053398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c1e4b958e4492eb96de9cc5d8d93a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.054037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c1e4b958e4492eb96de9cc5d8d93a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.060623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:35.072210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c1e4b958e4492eb96de9cc5d8d93a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.075496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c1e4b958e4492eb96de9cc5d8d93a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.091860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1019/2018] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:34 2022] DELETE /identity/v3/users/c9c1e4b958e4492eb96de9cc5d8d93a6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:35.096722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.097019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.097304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.115751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:35.116978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.151191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:35.182350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1000/2019] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:35.194034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.195490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.259702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea3764a-7ef0-4969-a693-912755a898ea admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.263325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.263979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.264264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.264545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.274725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.279542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846989-93a8-44a3-9159-d39b594ecf52 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.290985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1020/2020] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:35.414738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.416748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.449469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819578b-2c31-4675-88ea-fc2454d117dc None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.453527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.454716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.455078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.455392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] PATH_INFO: `/v3/projects/bba7d59ae92b4eaebe83a69815471696` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.457031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba7d59ae92b4eaebe83a69815471696)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.459147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.524614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba7d59ae92b4eaebe83a69815471696', 'request_id': 'req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.533334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba7d59ae92b4eaebe83a69815471696', 'request_id': 'req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.539967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba7d59ae92b4eaebe83a69815471696', 'request_id': 'req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.554486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] The token cache is being invalidate because project bba7d59ae92b4eaebe83a69815471696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:35.554782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.555561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.560339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1001/2021] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:35 2022] DELETE /identity/v3/projects/bba7d59ae92b4eaebe83a69815471696 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:35.562743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7626-b955-45b7-9fdb-7ace7c7725df None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.563154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7626-b955-45b7-9fdb-7ace7c7725df None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.563541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac7626-b955-45b7-9fdb-7ace7c7725df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.583285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-16ac7626-b955-45b7-9fdb-7ace7c7725df None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:35.584721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-16ac7626-b955-45b7-9fdb-7ace7c7725df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.601014 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.602479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.669941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1021/2022] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:35.678206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.678535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.678819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.699193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:35.700500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.704914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d5d6-0c64-4790-8aa6-a15b05ac788e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.710145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1022/2023] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:35.710315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.711228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.711646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.711933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] PATH_INFO: `/v3/projects/197b70c1fe7742ab9b23099858ffd558` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.715840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.715935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] RBAC: Authorizing `identity:delete_project(project_id=197b70c1fe7742ab9b23099858ffd558)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.718312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.730071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362c777-9892-4963-8e35-2e1bc3aafaf0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.735324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.736732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.737083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.737429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.738723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:35.739101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.742594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.757691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1023/2024] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:53:35.762248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.794841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64d7c-f067-46a4-928e-0467bb59c165 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:35.801359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197b70c1fe7742ab9b23099858ffd558', 'request_id': 'req-9359cd24-7075-4fda-8d2f-814fed6b541c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.809869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197b70c1fe7742ab9b23099858ffd558', 'request_id': 'req-9359cd24-7075-4fda-8d2f-814fed6b541c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.825815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197b70c1fe7742ab9b23099858ffd558', 'request_id': 'req-9359cd24-7075-4fda-8d2f-814fed6b541c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.827929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:35.859892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] The token cache is being invalidate because project 197b70c1fe7742ab9b23099858ffd558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:35.860380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.861819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:35.867799 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1002/2025] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:35 2022] DELETE /identity/v3/projects/197b70c1fe7742ab9b23099858ffd558 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:35.868325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.872228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.890638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:35.891426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.891821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.892169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.893054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:35.895797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec08b-b6b7-4f55-b658-3414cfb23527 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:35.901780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:35.944799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1024/2026] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:35 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:35.946555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:35.946975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:35.947415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:35.949619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1025/2027] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 17:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:53:35.950938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:35.953013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:35.963976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359cd24-7075-4fda-8d2f-814fed6b541c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.040931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.041956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.042390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.042749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] PATH_INFO: `/v3/users/63353655491941f09a0654844409790a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.044375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=63353655491941f09a0654844409790a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.046418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.049746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e00d-5495-4c13-bfd1-9c371a74ed82 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.054950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.055625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.055912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.056189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.056891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.058888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ee446-07ef-4656-8b5a-e25099e895e9 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.085872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1026/2028] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:35 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:36.085872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.085872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.085872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.114864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63353655491941f09a0654844409790a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.115621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63353655491941f09a0654844409790a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.122922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:36.124158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.131577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63353655491941f09a0654844409790a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.141529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63353655491941f09a0654844409790a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.142457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63353655491941f09a0654844409790a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.175519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1003/2029] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:35 2022] DELETE /identity/v3/users/63353655491941f09a0654844409790a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:36.176462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.178006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.212051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1027/2030] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:36.228838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.230671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.237336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:36.253169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1d6e8-3faa-4a48-ba3d-2c02cdb547aa None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.325099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccabf41-7747-4349-9887-1397eab92ebf admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.332384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.333141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.333564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.333848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.334758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.336881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.338909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.339640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.340001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.340395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.347549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:36.352147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1004/2031] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:36.353734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.356424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.358291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.363751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101cd85-abc0-4e54-b933-6df021862909 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.371702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.372400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.372738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.373240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.373349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1028/2032] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:36.375099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.377110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5ba5-c7f1-4b95-a71f-347b90e627e2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.392259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1005/2033] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:36.393538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.415123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78720451-c546-4242-b0d4-33f5eb81e490 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.422537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.423260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.423768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.424039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] PATH_INFO: `/v3/projects/ae269a9e9adb4f5a880d8e9670b366eb/users/4327967f546f4b49a43280c6e4fc3b8c/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.442200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae269a9e9adb4f5a880d8e9670b366eb, user_id=4327967f546f4b49a43280c6e4fc3b8c, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.445093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.467437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1029/2034] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:36 2022] PUT /identity/v3/projects/ae269a9e9adb4f5a880d8e9670b366eb/users/4327967f546f4b49a43280c6e4fc3b8c/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:36.478424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.478987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.478987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.508812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] MFA Rules not processed for user `4327967f546f4b49a43280c6e4fc3b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:36.510285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.518736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.520424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.601176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1006/2035] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:36.601809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f70e94-4cd9-4b86-86e2-0638bbd6a720 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.605462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.606527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.607415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.607698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.608010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] PATH_INFO: `/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.609188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19d27a1c42b4ecb87f6c6338b44a383)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.611203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.616491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e7b55-ccad-40bf-a0bd-3223d78d589e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.621236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.621877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.622163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.622498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.623179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.625081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.652445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1007/2036] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:36.656607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.661185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19d27a1c42b4ecb87f6c6338b44a383', 'request_id': 'req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.666858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19d27a1c42b4ecb87f6c6338b44a383', 'request_id': 'req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.674244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19d27a1c42b4ecb87f6c6338b44a383', 'request_id': 'req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.685589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] The token cache is being invalidate because project f19d27a1c42b4ecb87f6c6338b44a383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:36.685875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.686664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-440b46f3-f002-4f28-8a28-b6c7dfe4e4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:36.688226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1030/2037] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:36 2022] DELETE /identity/v3/projects/f19d27a1c42b4ecb87f6c6338b44a383 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:36.693408 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ff0b9-bbb9-4e70-a231-2bc9ca8c1183 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.693726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ff0b9-bbb9-4e70-a231-2bc9ca8c1183 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.694007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ff0b9-bbb9-4e70-a231-2bc9ca8c1183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.712864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-853ff0b9-bbb9-4e70-a231-2bc9ca8c1183 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:36.714140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-853ff0b9-bbb9-4e70-a231-2bc9ca8c1183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.719817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc766e-7549-4683-8083-c6240c06a033 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.722315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1031/2038] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:36.724553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.725222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.725509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.725781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.726506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.728565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.729793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.730085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.730386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.744189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:36.749040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:36.750266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.753048 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1008/2039] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:36.753955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.755562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.759327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1032/2040] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:36.763180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.768362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdaf7d-fd2b-4654-bc98-5c23d3c6fc6d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.775520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.776210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.776492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.776764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.777490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.778447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894341f-1966-4886-8f19-5f891ad91854 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.779260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.784571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.784571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.784571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.784958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.785582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:36.785889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.787518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.790390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1009/2041] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:36.795879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.800862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1033/2042] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:36.804930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.806678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd1ba-ddee-49a0-ba31-ba390541d05e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.811440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.812099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.812383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.812798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.813552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.815722 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.818539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996289c-441d-45d0-94ab-b3d2e29687a5 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.824026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.824606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.824889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.825536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.826262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.831823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1010/2043] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:36.832052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.832699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.843359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c86f0-fbfb-4f4c-afaf-894e24d6cc26 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.849194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.850088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.850498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.850668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] PATH_INFO: `/v3/projects/6eed3ba476ed4b1db5488f3f2faef9b6/users/201a00e852f1483fb0bea78097485354/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.866501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1034/2044] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:36.867468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:36.867832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eed3ba476ed4b1db5488f3f2faef9b6, user_id=201a00e852f1483fb0bea78097485354, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:36.870586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653ef3d-8901-449f-aa16-6032a265e908 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:36.881553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:36.885529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1011/2045] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:36 2022] PUT /identity/v3/projects/6eed3ba476ed4b1db5488f3f2faef9b6/users/201a00e852f1483fb0bea78097485354/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:36.896645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.897032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.897386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.923418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] MFA Rules not processed for user `201a00e852f1483fb0bea78097485354`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:36.925202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:36.935808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab1af-38c8-4ec4-abf9-57af8a5100d3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:36.997034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:36.997848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1012/2046] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:36.997987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:36.998193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:36.998560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:36.999249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.001535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.002640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.013489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106307c9-d1d1-410c-b9cb-7615256ac911 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.019025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.019731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.020077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.020560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.021173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.023184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.028455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1035/2047] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:36 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:37.029480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.040812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7283-6803-45c0-8388-c828f78211d6 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.045692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.046702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.046998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.047283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.048004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.050143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.052897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1013/2048] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:37.058446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.061709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1036/2049] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:37.065824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.076596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdfb86-fe1f-4695-a96a-3d60720d17df admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.131203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca62d80-6837-4fe1-b817-1236d50784a4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.136178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.136854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.137136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.137412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.138227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.140448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.141020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.141777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.142141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.142577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.143536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.145204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.156485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1037/2050] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:37.161113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.171747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b325bd-04a5-409b-9a51-7c33e49f5ac1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.177059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.177930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.178216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.178525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] PATH_INFO: `/v3/projects/a298f862286142ea81c9c052f88426e3/users/0eac4b627fd4467d8924b46582d2e93d/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.181960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1014/2051] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:37.183304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.193688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb278e5-5684-49b7-a8f7-83765f90e058 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.193821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a298f862286142ea81c9c052f88426e3, user_id=0eac4b627fd4467d8924b46582d2e93d, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.196502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24e3a9-70eb-465f-b33b-0ab8420c9ef6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.198531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.199173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.199448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.199725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.200447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.202105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.209557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:37.212755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1038/2052] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:37 2022] PUT /identity/v3/projects/a298f862286142ea81c9c052f88426e3/users/0eac4b627fd4467d8924b46582d2e93d/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:37.215889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1015/2053] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:37.224350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.224679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.225079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.227143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.239063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94151cf7-a392-4b4c-b6ae-ea5581f5174b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.245940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:37.258648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] MFA Rules not processed for user `0eac4b627fd4467d8924b46582d2e93d`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:37.259870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:37.302140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.303001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.303291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.303555 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] PATH_INFO: `/v3/projects/7345e09f2ffa42efb40c77a7fada26a1/users/124c10377a8945b8a075fa939a2ca188/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.323518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7345e09f2ffa42efb40c77a7fada26a1, user_id=124c10377a8945b8a075fa939a2ca188, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.326957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.328910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1039/2054] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:37.333772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.343403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1016/2055] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:37 2022] PUT /identity/v3/projects/7345e09f2ffa42efb40c77a7fada26a1/users/124c10377a8945b8a075fa939a2ca188/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:37.347145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d2399-a38f-4d00-bcc3-e8814401c581 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.348733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.411513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.412217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.412505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.412772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.413467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.415449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.416645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cffb71-39fc-448e-865d-11f7191722ca admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.422307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.423136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.424746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.424746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.426131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.428170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.439035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1017/2056] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:37.443151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.448011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1040/2057] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:37.453069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.457520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c40f0-772a-466d-9b48-9695d93abadc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.520294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327b667-bbb1-4234-bcfd-8459089c4b39 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.525025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.525667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.526132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.526132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.526490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.526614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.526901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.527174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] PATH_INFO: `/v3/projects/7345e09f2ffa42efb40c77a7fada26a1/users/124c10377a8945b8a075fa939a2ca188/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.527400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.529195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7345e09f2ffa42efb40c77a7fada26a1, user_id=124c10377a8945b8a075fa939a2ca188, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.529437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c3501-dab2-4380-9bb5-88e0e3012f3b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.531922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.544138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1018/2058] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:37 2022] PUT /identity/v3/projects/7345e09f2ffa42efb40c77a7fada26a1/users/124c10377a8945b8a075fa939a2ca188/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:37.552976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1041/2059] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:37.555881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.556343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.556767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.557059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.570960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:37.583488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:37.586663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] MFA Rules not processed for user `124c10377a8945b8a075fa939a2ca188`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:37.588604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:37.626600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93c332-df97-44fe-9986-7331c215f2e1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.632335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.633418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.633706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.634248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.635064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.637536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.649647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1019/2060] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:37.653734 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.663893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1042/2061] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:37.670296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d16d2-9267-4fe7-a75e-8e244bedef16 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.676528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.676668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.677075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.677416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.677528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:37.677826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] PATH_INFO: `/v3/projects/8029e671c9274d319bc022c916ee6482/users/f2f1c10688504dc8ba4d57efa2b28f2a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.697015 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8029e671c9274d319bc022c916ee6482, user_id=f2f1c10688504dc8ba4d57efa2b28f2a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.700522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.718743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1020/2062] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:37 2022] PUT /identity/v3/projects/8029e671c9274d319bc022c916ee6482/users/f2f1c10688504dc8ba4d57efa2b28f2a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:37.723218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.748190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932fa4d-39d8-4002-9d90-d324196b3a7d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.791423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0e90a-c35a-4cac-961e-b8a441e65d29 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.795983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.796756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.797036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.797314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.798030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.800224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.810075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1021/2063] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:37.813673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.813867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.814921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.815208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.815481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.817268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:37.824663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbd630-c77d-4039-89f8-367f22f4f4af admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:37.829365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:37.830241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.830545 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.830817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] PATH_INFO: `/v3/projects/8029e671c9274d319bc022c916ee6482/users/f2f1c10688504dc8ba4d57efa2b28f2a/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.832779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8029e671c9274d319bc022c916ee6482, user_id=f2f1c10688504dc8ba4d57efa2b28f2a, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.835451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65a6f4-b124-4ed4-8431-b71a572e2a31 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.845658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1022/2064] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:37 2022] PUT /identity/v3/projects/8029e671c9274d319bc022c916ee6482/users/f2f1c10688504dc8ba4d57efa2b28f2a/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:37.856402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:37.856929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:37.857138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:37.881052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:37.882789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:37.892201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] MFA Rules not processed for user `f2f1c10688504dc8ba4d57efa2b28f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:37.893458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:37.894219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1043/2065] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:37.962711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:37.973134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1023/2066] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:37.989177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:37.990907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:38.005240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990033b-5913-4c96-bae7-679f78595e79 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:38.010257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:38.062038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:38.062713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:38.062993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:38.063283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:38.073447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:38.074916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:38.084799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1044/2067] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:38.197262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:38.208582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72369abc-978f-4a56-9ffa-f6c0bc0e0bce None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:38.212345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:38.213094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:38.213511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:38.213865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:38.280673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:38.282349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20cd9a-b6fe-4996-81ce-1e3820fd6dd9 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:38.291693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1024/2068] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:38.552342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:38.617123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0f64-e472-4e6e-bdc1-e760fb2bde90 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:38.621273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:38.622300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:38.622615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:38.623005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] PATH_INFO: `/v3/users/0eac4b627fd4467d8924b46582d2e93d` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:38.624192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eac4b627fd4467d8924b46582d2e93d)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:38.626102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:38.673255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eac4b627fd4467d8924b46582d2e93d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:38.686881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eac4b627fd4467d8924b46582d2e93d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:38.687516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eac4b627fd4467d8924b46582d2e93d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:38.700516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eac4b627fd4467d8924b46582d2e93d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:38.701178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eac4b627fd4467d8924b46582d2e93d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:38.713239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1045/2069] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:38 2022] DELETE /identity/v3/users/0eac4b627fd4467d8924b46582d2e93d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:38.717638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:38.718087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:38.718198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:38.736673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:38.737859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:38.796860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1025/2070] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:38.807284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:38.808909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:38.844998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:38.872148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2248249-a9c6-4a0c-a83f-963609d14830 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:38.875763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:38.876405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:38.876689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:38.876962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:38.887265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:38.889608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:38.900069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1046/2071] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:39.191374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:39.192792 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:39.209687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686b597-ce22-4a17-bc52-9553556422ba None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:39.214502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:39.215352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:39.215644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:39.215913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] PATH_INFO: `/v3/projects/a298f862286142ea81c9c052f88426e3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:39.217088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a298f862286142ea81c9c052f88426e3)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:39.219064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:39.266716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a298f862286142ea81c9c052f88426e3', 'request_id': 'req-13dc0063-6fb6-46e8-8289-cfad012fcbb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.273405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a298f862286142ea81c9c052f88426e3', 'request_id': 'req-13dc0063-6fb6-46e8-8289-cfad012fcbb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.278783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a298f862286142ea81c9c052f88426e3', 'request_id': 'req-13dc0063-6fb6-46e8-8289-cfad012fcbb3'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.291596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] The token cache is being invalidate because project a298f862286142ea81c9c052f88426e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:39.291889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.292521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.294102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1026/2072] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:39 2022] DELETE /identity/v3/projects/a298f862286142ea81c9c052f88426e3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:39.298030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:39.307595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3516a3-9d01-427d-8260-23ef3f44494b service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:39.363191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:39.364545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:39.364832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:39.365108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] PATH_INFO: `/v3/users/f2f1c10688504dc8ba4d57efa2b28f2a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:39.366367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f1c10688504dc8ba4d57efa2b28f2a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:39.368984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:39.424928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f1c10688504dc8ba4d57efa2b28f2a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.438223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f1c10688504dc8ba4d57efa2b28f2a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.438223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f1c10688504dc8ba4d57efa2b28f2a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.451815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f1c10688504dc8ba4d57efa2b28f2a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.452565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-65d2760a-2fc7-4b0e-b7e9-d8ad46d5378e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f1c10688504dc8ba4d57efa2b28f2a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.466701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1047/2073] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:39 2022] DELETE /identity/v3/users/f2f1c10688504dc8ba4d57efa2b28f2a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:39.740515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:39.742002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:39.759658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0063-6fb6-46e8-8289-cfad012fcbb3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:39.773748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:39.822960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:39.824124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:39.824406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:39.824676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] PATH_INFO: `/v3/projects/8029e671c9274d319bc022c916ee6482` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:39.826030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8029e671c9274d319bc022c916ee6482)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:39.827989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:39.874714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8029e671c9274d319bc022c916ee6482', 'request_id': 'req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.881586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8029e671c9274d319bc022c916ee6482', 'request_id': 'req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.886955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8029e671c9274d319bc022c916ee6482', 'request_id': 'req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.899521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] The token cache is being invalidate because project 8029e671c9274d319bc022c916ee6482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:39.899838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.900448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f1a837d9-7b67-4c03-b153-41dfb2cfb50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:39.901984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1027/2074] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:39 2022] DELETE /identity/v3/projects/8029e671c9274d319bc022c916ee6482 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:39.908163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:39.908451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:39.908725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:39.927198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:39.928430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:39.984488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:40.005264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1048/2075] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:40.012860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.013151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.013430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.033236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:40.036314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:40.047927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1028/2076] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:40.049886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.060294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95ee17-8976-42ba-b43e-90ad6cc0027c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.066080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.066839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.067188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.067538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.068328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:40.068710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.070434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.085320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1049/2077] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:40.089750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.099416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88278a-40ed-4186-a649-fb32816a9b19 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.106223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.107021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.107304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.107583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.108289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.110251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.137253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1029/2078] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:40 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:40.141339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.151752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e26b-573f-4d09-b64d-3e9a4fccc600 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.207522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.208107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.208402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.208679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.209368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.211336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.231459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1050/2079] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:40 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:40.235606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.246611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae333-2c4d-4f11-a1c7-44fc74805b06 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.251328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.252707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.252986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.253258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.254013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.255786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.269689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1030/2080] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:40.271259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.281872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325614b7-b5fd-4e79-bce9-1a0199b1b268 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.286912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.287584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.287869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.288144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.288870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.290545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f1fda-70c8-4e31-b3ae-8f16e0e0d940 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.307889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1051/2081] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:40.312019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.322651 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df7d6-9599-46bd-ba04-cbc1f27a7f06 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.327470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.328335 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.328618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.328895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] PATH_INFO: `/v3/projects/537911eafd204734a304a41bf8fde679/users/de89581b05ea4cada13321ea59b54402/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.343425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537911eafd204734a304a41bf8fde679, user_id=de89581b05ea4cada13321ea59b54402, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.346642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.358714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1031/2082] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:40 2022] PUT /identity/v3/projects/537911eafd204734a304a41bf8fde679/users/de89581b05ea4cada13321ea59b54402/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:40.370825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.371115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.371394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.398201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] MFA Rules not processed for user `de89581b05ea4cada13321ea59b54402`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:40.399446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:40.467563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1052/2083] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:40.480527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:40.482836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:40.557112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:40.579608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c42839-8ae4-48ba-855a-788108109493 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:40.586733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:40.587473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:40.587829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:40.588173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:40.601605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:40.604115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:40.620286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1032/2084] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:41.626515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:41.629876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:41.698574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:41.703043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bafefe-d998-4111-9f84-1d5019505559 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:41.715633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:41.716297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:41.716582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:41.716847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:41.719256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:41.789708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:41.791154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:41.801247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1053/2085] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:42.516976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:42.547610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0e525-1c74-41b5-a6fd-5d434658f6a6 service glance] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:42.559681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:42.560770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:42.561149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:42.561517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:42.564617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:42.615013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:42.616206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:42.644123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:42.645563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d6029-fc9d-4e42-be60-72c557b5efb0 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:42.655435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1033/2086] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:42.702568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19352a0-0a18-407e-9e9d-02026b1371d2 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:42.709512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:42.710498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:42.711068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:42.711225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] PATH_INFO: `/v3/users/822c13c8722c40628e2e972b9e5b4283` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:42.712707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] RBAC: Authorizing `identity:delete_user(user_id=822c13c8722c40628e2e972b9e5b4283)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:42.714755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:42.793275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822c13c8722c40628e2e972b9e5b4283'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:42.807249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822c13c8722c40628e2e972b9e5b4283'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:42.807956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822c13c8722c40628e2e972b9e5b4283'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:42.821429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822c13c8722c40628e2e972b9e5b4283'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:42.822097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822c13c8722c40628e2e972b9e5b4283'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:42.836291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1054/2087] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:42 2022] DELETE /identity/v3/users/822c13c8722c40628e2e972b9e5b4283 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:42.840652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:42.840944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:42.841224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:42.859196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:42.860439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:42.920733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1034/2088] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:42.931128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:42.932561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:42.992185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:43.038523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1b24d-8cd3-4b4e-acda-aa5bc971247b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:43.042259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:43.042967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:43.043257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:43.043820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:43.053693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:43.055164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:43.067033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1055/2089] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:43.350991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:43.350991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:43.371762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36deb7-db08-4df9-85b0-378483c6a7d0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:43.377183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:43.378125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:43.378462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:43.379456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] PATH_INFO: `/v3/projects/a00ba85fadb54b82b8f08c17fc1810b9` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:43.380225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00ba85fadb54b82b8f08c17fc1810b9)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:43.382238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:43.453865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00ba85fadb54b82b8f08c17fc1810b9', 'request_id': 'req-63738e7e-ef46-4dbf-a672-000f662761c4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.464183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00ba85fadb54b82b8f08c17fc1810b9', 'request_id': 'req-63738e7e-ef46-4dbf-a672-000f662761c4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.475122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00ba85fadb54b82b8f08c17fc1810b9', 'request_id': 'req-63738e7e-ef46-4dbf-a672-000f662761c4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.544248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] The token cache is being invalidate because project a00ba85fadb54b82b8f08c17fc1810b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:43.544248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.554084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.559288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1035/2090] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:43 2022] DELETE /identity/v3/projects/a00ba85fadb54b82b8f08c17fc1810b9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:43.563191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:43.576324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaffb1f-b2ec-4159-97a6-d6be3461a443 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:43.724071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:43.724935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:43.725252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:43.725584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] PATH_INFO: `/v3/users/c2b85c53b9444d5e99149de6b02cba00` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:43.727199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b85c53b9444d5e99149de6b02cba00)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:43.729403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:43.847409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b85c53b9444d5e99149de6b02cba00'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.863287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b85c53b9444d5e99149de6b02cba00'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.864323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b85c53b9444d5e99149de6b02cba00'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.880011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b85c53b9444d5e99149de6b02cba00'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.881071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ee1f4922-2684-4b56-baf6-3a6ff3b120f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b85c53b9444d5e99149de6b02cba00'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:43.897646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1056/2091] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:43 2022] DELETE /identity/v3/users/c2b85c53b9444d5e99149de6b02cba00 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:44.429095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:44.430778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:44.447040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:44.461711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63738e7e-ef46-4dbf-a672-000f662761c4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:44.549757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:44.550887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:44.551161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:44.551551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] PATH_INFO: `/v3/projects/7a0ffdda57a64e71b22980c48a492bcb` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:44.553080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0ffdda57a64e71b22980c48a492bcb)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:44.555388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:44.617356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0ffdda57a64e71b22980c48a492bcb', 'request_id': 'req-7ba84349-76e5-4e7f-bf9d-249cde87a314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:44.618962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0ffdda57a64e71b22980c48a492bcb', 'request_id': 'req-7ba84349-76e5-4e7f-bf9d-249cde87a314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:44.624524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0ffdda57a64e71b22980c48a492bcb', 'request_id': 'req-7ba84349-76e5-4e7f-bf9d-249cde87a314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:44.639131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] The token cache is being invalidate because project 7a0ffdda57a64e71b22980c48a492bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:44.639426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:44.640044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7ba84349-76e5-4e7f-bf9d-249cde87a314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:44.641646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1036/2092] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:44 2022] DELETE /identity/v3/projects/7a0ffdda57a64e71b22980c48a492bcb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:44.646661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:44.646957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:44.647272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:44.665467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:44.666747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:44.724194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:44.758672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1057/2093] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:44.767113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:44.767536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:44.767785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:44.793020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:44.793020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:44.819729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1037/2094] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:44.824832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:44.840352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50441a-17ef-4ce0-8614-50ee51addc1a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:44.845668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:44.846566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:44.846844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:44.847273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:44.848012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:44.848323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:44.849937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:44.863149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1058/2095] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:53:44.868343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:44.890399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9c1d6-7b43-48d0-bf10-ffb9820f80e4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:44.895769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:44.896452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:44.896826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:44.897179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:44.898044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:44.900092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:44.926742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1038/2096] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:44 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:44.932784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:44.943753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179f11-0347-4ab4-b8d1-ea594314429f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.009026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.009817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.010030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.010372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.011103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.013915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.037887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1059/2097] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:44 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:45.042603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.053716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6bb9e-bab7-426a-8850-d17352c05b9c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.058252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.058943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.059220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.059491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.060204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.061833 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.072919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1039/2098] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:45.077573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.087599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100de0d-e5ce-404c-a732-e027bcee191e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.092395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.092992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.093333 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.093606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.094393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.096504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387040ed-80a2-4055-90aa-cf1b12adfc93 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.108080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1060/2099] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:45.113365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.125170 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b88c3-5adf-4efd-82fd-0a845b10ee2e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.130591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.131612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.132021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.132284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] PATH_INFO: `/v3/projects/83af7caeba0a4623abb6f0acfa8cd3e6/users/45c423c8c6434ba284b6206df71f69c1/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.147685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=83af7caeba0a4623abb6f0acfa8cd3e6, user_id=45c423c8c6434ba284b6206df71f69c1, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.150360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.164422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1040/2100] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:45 2022] PUT /identity/v3/projects/83af7caeba0a4623abb6f0acfa8cd3e6/users/45c423c8c6434ba284b6206df71f69c1/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:45.175337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.175736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.176090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.202740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] MFA Rules not processed for user `45c423c8c6434ba284b6206df71f69c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:45.203989 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:45.269877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1061/2101] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:45.274779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.305953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:45.340976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5fbc-45d1-4c15-a6d2-f7b9ca5df69c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.346720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.347316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.347604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.347887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.348851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.350870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.375855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1041/2102] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:45.381910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.399922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b36258-2ee0-4e7d-be3d-cc581b95c8b8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.441493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:45.465739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.466433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.466719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.466994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.467678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.469617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.494926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1062/2103] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:45.497172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.508173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a63eb-ed41-4058-89a7-fb911760c9f6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.515609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.516334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.516684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.517564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.518524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.520258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.531104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1042/2104] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:45.535754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.558750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a4c1-4e35-40d4-96d4-25e34c8c0d5e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.565356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.566032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.566312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.566615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.567583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.569049 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42553e4-fc84-458d-b884-f563e29f1e08 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.589023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1063/2105] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:45.590483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.603075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601fbdc-53ce-4de1-b74e-0a558b845e2a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.608530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.609448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.609728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.610032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] PATH_INFO: `/v3/projects/075c55672a194d419bb114ce5bfb4e0d/users/08d6a834c388437997c10f68002debe7/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.628490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=075c55672a194d419bb114ce5bfb4e0d, user_id=08d6a834c388437997c10f68002debe7, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.631198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.651032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1043/2106] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:45 2022] PUT /identity/v3/projects/075c55672a194d419bb114ce5bfb4e0d/users/08d6a834c388437997c10f68002debe7/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:45.662419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.662944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.663522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.699799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] MFA Rules not processed for user `08d6a834c388437997c10f68002debe7`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:45.701016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:45.774458 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1064/2107] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:45.779108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.844325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd64ef-44a1-42fa-ac44-362094b9b60b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.849732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.850383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.850643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.850945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.853338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.853561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:45.888184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1044/2108] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:45.891646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:45.903415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf827fb-f540-4272-a916-ea57e5fd9ef8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:45.987166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:45.989167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:45.990121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:45.990805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:45.993721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:45.997856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.036169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1065/2109] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:45 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:53:46.040559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:46.054052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d91a-0642-48b4-b4d2-e949d7f0527f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:46.062476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:46.063307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.063823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.064766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.065605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:46.067380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.085279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1045/2110] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:46.086777 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:46.100701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5496d3-dbc6-4b04-9df3-1d863c888ea0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:46.113042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:46.116171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.116457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.116660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] PATH_INFO: `/v3/projects/d201fe06026244d5b176b68db54ee38f/users/fd4a5a41e68a41eab038e72c0254ade5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.148789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d201fe06026244d5b176b68db54ee38f, user_id=fd4a5a41e68a41eab038e72c0254ade5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:46.151591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.166673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1066/2111] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:46 2022] PUT /identity/v3/projects/d201fe06026244d5b176b68db54ee38f/users/fd4a5a41e68a41eab038e72c0254ade5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:46.170853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:46.183240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3f0d6-16f3-4ff0-8753-d140d043fbd6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:46.235549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:46.247068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:46.247808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.248468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.248468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.249307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:46.251039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb0bda-607c-4ff1-832b-bf3343e4fbe5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.266618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1046/2112] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:46.267904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:46.282112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf8517-44b4-4065-ad30-584960c07b42 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:46.287877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:46.288842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.289124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.289471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] PATH_INFO: `/v3/projects/d201fe06026244d5b176b68db54ee38f/users/fd4a5a41e68a41eab038e72c0254ade5/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.292763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d201fe06026244d5b176b68db54ee38f, user_id=fd4a5a41e68a41eab038e72c0254ade5, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:46.296722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.306344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:46.311149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1067/2113] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:46 2022] PUT /identity/v3/projects/d201fe06026244d5b176b68db54ee38f/users/fd4a5a41e68a41eab038e72c0254ade5/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:46.326861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.327183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.327439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.377362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] MFA Rules not processed for user `fd4a5a41e68a41eab038e72c0254ade5`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:46.378970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:46.477125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1047/2114] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:46.495469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:46.498405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:46.559426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34db06-17dd-4b79-8681-47a86f93b01b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:46.563236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:46.564487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:46.564794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:46.565163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:46.620387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:46.621758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:46.631549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1068/2115] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:48.233234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:48.243187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a1f37-d5fe-4bcc-b97e-003d67ef154a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:48.248135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:48.249175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:48.249609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:48.249962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:48.319175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:48.321699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2e1b9-3c4c-4874-92c8-51050c2db987 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:48.335796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1048/2116] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:49.221281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:49.237511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:49.318492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c16-4b6c-4e79-bfbd-b649fed92371 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:49.325936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:49.326997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:49.327348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:49.327747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] PATH_INFO: `/v3/users/4327967f546f4b49a43280c6e4fc3b8c` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:49.329083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4327967f546f4b49a43280c6e4fc3b8c)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:49.331219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:49.441562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4327967f546f4b49a43280c6e4fc3b8c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:49.472204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4327967f546f4b49a43280c6e4fc3b8c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:49.473075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4327967f546f4b49a43280c6e4fc3b8c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:49.477060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:49.503938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4327967f546f4b49a43280c6e4fc3b8c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:49.504698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4327967f546f4b49a43280c6e4fc3b8c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:49.530048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1069/2117] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:49 2022] DELETE /identity/v3/users/4327967f546f4b49a43280c6e4fc3b8c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:49.530488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:49.530488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:49.530488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:49.596378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:49.596378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:49.703140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1049/2118] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:49.711975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:49.713425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:49.786348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:49.787910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:49.824872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:49.825422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5a0f0-1f63-41c3-82aa-2b7ac3b0ba34 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:49.834795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:49.835811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:49.836290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:49.836907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:49.849624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:49.851121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:49.863736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1070/2119] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:49.898563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f80cfc-b901-4131-80c6-94b27bd872f3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:49.903403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:49.904284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:49.904873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:49.905319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:49.907397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:50.021037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:50.022876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:50.047234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1050/2120] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:50.476495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:50.478071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:50.508664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750295-7c20-46ca-8a1c-605f202b8245 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:50.517562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:50.520318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:50.520585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:50.520871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] PATH_INFO: `/v3/projects/ae269a9e9adb4f5a880d8e9670b366eb` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:50.522309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae269a9e9adb4f5a880d8e9670b366eb)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:50.524403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:50.603700 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae269a9e9adb4f5a880d8e9670b366eb', 'request_id': 'req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.610278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae269a9e9adb4f5a880d8e9670b366eb', 'request_id': 'req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.625235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae269a9e9adb4f5a880d8e9670b366eb', 'request_id': 'req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.644162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] The token cache is being invalidate because project ae269a9e9adb4f5a880d8e9670b366eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:50.646121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.646941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.656229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1071/2121] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:50 2022] DELETE /identity/v3/projects/ae269a9e9adb4f5a880d8e9670b366eb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:50.656787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:50.678417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed590a-fef6-4d23-bdfa-e410d7606b0c service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:50.830890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:50.832292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:50.832581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:50.832857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] PATH_INFO: `/v3/users/201a00e852f1483fb0bea78097485354` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:50.835138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201a00e852f1483fb0bea78097485354)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:50.848082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:50.987820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201a00e852f1483fb0bea78097485354'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:50.988550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201a00e852f1483fb0bea78097485354'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.017454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201a00e852f1483fb0bea78097485354'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.019900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:51.029705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201a00e852f1483fb0bea78097485354'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.030388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201a00e852f1483fb0bea78097485354'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.076538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1051/2122] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:50 2022] DELETE /identity/v3/users/201a00e852f1483fb0bea78097485354 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:51.422751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:51.424243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:51.459973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903a5c7-7a10-43b1-b897-f56a5c3d1fed admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:51.533674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:51.535010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:51.535336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:51.535610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] PATH_INFO: `/v3/projects/6eed3ba476ed4b1db5488f3f2faef9b6` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:51.536902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eed3ba476ed4b1db5488f3f2faef9b6)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:51.538898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:51.540577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:51.613256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eed3ba476ed4b1db5488f3f2faef9b6', 'request_id': 'req-08b9ebe5-03c0-4937-8461-175f79ca3e17'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.619235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eed3ba476ed4b1db5488f3f2faef9b6', 'request_id': 'req-08b9ebe5-03c0-4937-8461-175f79ca3e17'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.629519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eed3ba476ed4b1db5488f3f2faef9b6', 'request_id': 'req-08b9ebe5-03c0-4937-8461-175f79ca3e17'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.644053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] The token cache is being invalidate because project 6eed3ba476ed4b1db5488f3f2faef9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:51.644419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.645176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.646874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1072/2123] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:51 2022] DELETE /identity/v3/projects/6eed3ba476ed4b1db5488f3f2faef9b6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:51.650984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:51.652407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:51.665229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2baeb3-6dba-4fe1-983e-d4ba2133e0e1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:51.726972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:51.727856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:51.728141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:51.728422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] PATH_INFO: `/v3/users/124c10377a8945b8a075fa939a2ca188` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:51.729731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=124c10377a8945b8a075fa939a2ca188)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:51.731582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:51.783318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124c10377a8945b8a075fa939a2ca188'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.784082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124c10377a8945b8a075fa939a2ca188'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.796723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124c10377a8945b8a075fa939a2ca188'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.804874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124c10377a8945b8a075fa939a2ca188'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.805503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d156cad3-7c4b-4a1c-af3e-fe711893f74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124c10377a8945b8a075fa939a2ca188'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:51.818444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1052/2124] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:51 2022] DELETE /identity/v3/users/124c10377a8945b8a075fa939a2ca188 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:52.115126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.116900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:52.135522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ebe5-03c0-4937-8461-175f79ca3e17 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.195528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.196399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.196735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.197009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] PATH_INFO: `/v3/projects/7345e09f2ffa42efb40c77a7fada26a1` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.198523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7345e09f2ffa42efb40c77a7fada26a1)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.200453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.254645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7345e09f2ffa42efb40c77a7fada26a1', 'request_id': 'req-2983676e-dc21-43d9-83e2-87ad8f70bc6f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:52.260809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7345e09f2ffa42efb40c77a7fada26a1', 'request_id': 'req-2983676e-dc21-43d9-83e2-87ad8f70bc6f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:52.270456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7345e09f2ffa42efb40c77a7fada26a1', 'request_id': 'req-2983676e-dc21-43d9-83e2-87ad8f70bc6f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:52.283853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] The token cache is being invalidate because project 7345e09f2ffa42efb40c77a7fada26a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:53:52.284262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:52.284940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2983676e-dc21-43d9-83e2-87ad8f70bc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:52.286799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1073/2125] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:53:52 2022] DELETE /identity/v3/projects/7345e09f2ffa42efb40c77a7fada26a1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:52.291704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.292279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.292700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.312824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:52.314738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:52.378570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1053/2126] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:52.386309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.386635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.386917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.406378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:52.408142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:52.416074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1074/2127] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:52.420330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.425727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:52.431283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c24d8-161b-4b81-a040-f476417e6bfd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.439616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.440332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.440619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.441211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.442534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:53:52.443198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.444879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.463316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1054/2128] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:53:52.467677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.479054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bcb46-8974-4638-9eb4-ab36f6f5696f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.483854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.484500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.484811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.485080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.486433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.488326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.499072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:52.517289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1075/2129] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:53:52 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:52.521252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.531125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47ce2-c5f7-488a-8962-3b7494ceaefe admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.589406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.590678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.592402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.592402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.592402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.594222 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.624132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1055/2130] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:53:52 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:53:52.625733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.639332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbbe48-7c44-4c92-b8b9-a5b23ddb84b1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.646734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.647680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.648033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.648377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.649243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.650971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.663172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1076/2131] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:53:52.666844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.677642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483fcc4-07a7-484b-89c5-2ad9cef658cb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.682362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.683013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.683296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.683573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.684294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.685872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f7538-d6e6-4273-a5cb-aa8d3b5d2758 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.695928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1056/2132] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:53:52.700110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.710582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cc176-8653-4727-b76b-9358fa3933d1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.715388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.716491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.716770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.717041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] PATH_INFO: `/v3/projects/76e561c925824ebcb257686f304a1c1c/users/55c5ce0fa03e4c50aa5cc8b64d7ccc73/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.732135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e561c925824ebcb257686f304a1c1c, user_id=55c5ce0fa03e4c50aa5cc8b64d7ccc73, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.734838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.747268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1077/2133] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:53:52 2022] PUT /identity/v3/projects/76e561c925824ebcb257686f304a1c1c/users/55c5ce0fa03e4c50aa5cc8b64d7ccc73/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:52.759113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.759486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.759843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.789236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] MFA Rules not processed for user `55c5ce0fa03e4c50aa5cc8b64d7ccc73`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:52.790469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:52.852054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1057/2134] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:52.863534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:52.865242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:52.921626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4570c83-4b25-4711-b054-f99cb4d2298d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:52.925261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:52.925903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:52.926593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:52.926593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:52.936229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:52.937762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:52.946796 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1078/2135] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:53:59.348130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:59.349672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.401995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:59.405754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.431873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:59.436188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3192068-ed5f-4f84-9345-7f27a1618e14 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:59.442490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:59.443355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.443634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.443908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] PATH_INFO: `/v3/users/4ab4e3ebbe44446cb34dd850b0dcf2bc` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.445110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab4e3ebbe44446cb34dd850b0dcf2bc)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:59.447061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:59.479312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b4501-bbde-43b4-829d-dd790b875f80 service glance] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:59.484746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:59.485616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.485903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.486180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] PATH_INFO: `/v3/users/45c423c8c6434ba284b6206df71f69c1` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.488185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c423c8c6434ba284b6206df71f69c1)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:59.490121 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:59.496952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab4e3ebbe44446cb34dd850b0dcf2bc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.497649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab4e3ebbe44446cb34dd850b0dcf2bc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.510911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab4e3ebbe44446cb34dd850b0dcf2bc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.518807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab4e3ebbe44446cb34dd850b0dcf2bc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.519678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1d90a9bb-d0c3-4968-94cb-235c76535b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab4e3ebbe44446cb34dd850b0dcf2bc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.532578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1058/2136] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:59 2022] DELETE /identity/v3/users/4ab4e3ebbe44446cb34dd850b0dcf2bc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:53:59.537068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.537364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.537647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.540380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c423c8c6434ba284b6206df71f69c1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.548080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c423c8c6434ba284b6206df71f69c1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.549238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c423c8c6434ba284b6206df71f69c1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.555063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:59.555486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:59.556745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.563949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c423c8c6434ba284b6206df71f69c1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.565304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-f9a41c26-c489-4f15-9096-40034608bdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c423c8c6434ba284b6206df71f69c1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:53:59.580770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1079/2137] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:53:59 2022] DELETE /identity/v3/users/45c423c8c6434ba284b6206df71f69c1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:53:59.585251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.585625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.585873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.602633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:53:59.603958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.622768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1059/2138] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:53:59.634155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:59.642067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.723769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1080/2139] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:53:59.730905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:53:59.732833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:53:59.759196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843425-cbfe-4149-8b2f-1819fee12a75 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:59.812826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:53:59.838158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee205c-bac9-4f1b-a191-6634bb85f822 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:53:59.846486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:59.851598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.851775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.851938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.857596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:53:59.858253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:53:59.858564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:53:59.858929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:53:59.876000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:59.878059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:59.889781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:53:59.891332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf56dafc-9507-4135-a2cb-69dafbf7f7c9 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:53:59.901851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1081/2140] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:53:59.911188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1060/2141] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:00.159969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.161762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.176840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d30137-00d9-4d43-8679-29a0c91b54ec service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.182448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.183319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.183601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.184245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] PATH_INFO: `/v3/users/bc6b7f1dc44548cbbff7121bb6f52801` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.185203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6b7f1dc44548cbbff7121bb6f52801)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.187165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:00.267542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6b7f1dc44548cbbff7121bb6f52801'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.276200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6b7f1dc44548cbbff7121bb6f52801'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.276889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6b7f1dc44548cbbff7121bb6f52801'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.293159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6b7f1dc44548cbbff7121bb6f52801'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.294110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6b7f1dc44548cbbff7121bb6f52801'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.299829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:00.314144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1082/2142] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/users/bc6b7f1dc44548cbbff7121bb6f52801 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:00.316390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.316683 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.317011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.338532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:00.339836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.417868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1061/2143] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:00.428782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.430495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.438032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.440892 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.462083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2753bcd-b465-4be1-95a9-3b28e045b2ec None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.470774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.472276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.472628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.472970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] PATH_INFO: `/v3/projects/83af7caeba0a4623abb6f0acfa8cd3e6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.483429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83af7caeba0a4623abb6f0acfa8cd3e6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.487877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:00.514544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d25b-ba0a-4eee-9150-bea344941e9e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.518376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.519768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.520253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.520635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.531982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.534029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:00.545290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1083/2144] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:00.546205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.548484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.555336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83af7caeba0a4623abb6f0acfa8cd3e6', 'request_id': 'req-6bc9c9f7-dc66-4585-83dc-17b355ec5365'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.562536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83af7caeba0a4623abb6f0acfa8cd3e6', 'request_id': 'req-6bc9c9f7-dc66-4585-83dc-17b355ec5365'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.566364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb028fe-5cc7-4451-acf0-369a81158b46 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.569680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83af7caeba0a4623abb6f0acfa8cd3e6', 'request_id': 'req-6bc9c9f7-dc66-4585-83dc-17b355ec5365'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.589528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] The token cache is being invalidate because project 83af7caeba0a4623abb6f0acfa8cd3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:00.589927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.590800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.592840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1062/2145] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/projects/83af7caeba0a4623abb6f0acfa8cd3e6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:00.603944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.605607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.672423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.673332 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.674384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.674678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] PATH_INFO: `/v3/projects/9d599fc15c1c4b1e9419484a43c5298a` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.675946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d599fc15c1c4b1e9419484a43c5298a)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.678033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:00.723447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c9f7-dc66-4585-83dc-17b355ec5365 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.729691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.731019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.731902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.732370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] PATH_INFO: `/v3/users/08d6a834c388437997c10f68002debe7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.734208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d6a834c388437997c10f68002debe7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.737277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:00.739417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:00.756220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d599fc15c1c4b1e9419484a43c5298a', 'request_id': 'req-c923f7d2-21ca-4385-b0fb-b2bbdae09882'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.763000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d599fc15c1c4b1e9419484a43c5298a', 'request_id': 'req-c923f7d2-21ca-4385-b0fb-b2bbdae09882'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.773621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d599fc15c1c4b1e9419484a43c5298a', 'request_id': 'req-c923f7d2-21ca-4385-b0fb-b2bbdae09882'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.793404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] The token cache is being invalidate because project 9d599fc15c1c4b1e9419484a43c5298a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:00.793739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.794567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.796742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1084/2146] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/projects/9d599fc15c1c4b1e9419484a43c5298a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:00.800566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.802190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.831614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d6a834c388437997c10f68002debe7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.832360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d6a834c388437997c10f68002debe7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.840409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:00.856711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d6a834c388437997c10f68002debe7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.870775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d6a834c388437997c10f68002debe7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.871494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d6a834c388437997c10f68002debe7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:00.890168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1063/2147] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/users/08d6a834c388437997c10f68002debe7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:00.905905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923f7d2-21ca-4385-b0fb-b2bbdae09882 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:00.925442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:00.927649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:00.979540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:00.980431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:00.980848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:00.980968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] PATH_INFO: `/v3/users/9c0ab1a89a264113985bb8757fa40677` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:00.982696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0ab1a89a264113985bb8757fa40677)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:00.985513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:01.066003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0ab1a89a264113985bb8757fa40677'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.076286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43b923-1af9-4a52-8951-8d71a43d47c2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:01.080003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0ab1a89a264113985bb8757fa40677'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.081340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0ab1a89a264113985bb8757fa40677'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.084002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:01.085314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:01.085985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:01.086539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] PATH_INFO: `/v3/projects/f770c75aaa8e40098c18a21c8d4511be` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:01.088474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f770c75aaa8e40098c18a21c8d4511be)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:01.090786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:01.111945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0ab1a89a264113985bb8757fa40677'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.112859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0ab1a89a264113985bb8757fa40677'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.128424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1085/2148] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/users/9c0ab1a89a264113985bb8757fa40677 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:01.148971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f770c75aaa8e40098c18a21c8d4511be', 'request_id': 'req-bd52eecf-f939-447e-939d-e0255d92a072'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.156050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f770c75aaa8e40098c18a21c8d4511be', 'request_id': 'req-bd52eecf-f939-447e-939d-e0255d92a072'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.168506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f770c75aaa8e40098c18a21c8d4511be', 'request_id': 'req-bd52eecf-f939-447e-939d-e0255d92a072'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.186932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] The token cache is being invalidate because project f770c75aaa8e40098c18a21c8d4511be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:01.187221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.188511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.191280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1064/2149] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:00 2022] DELETE /identity/v3/projects/f770c75aaa8e40098c18a21c8d4511be => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:01.194874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:01.196424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:01.254969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:01.270074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08f2d9-8557-4abb-b7f8-dc9d863dc74b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:01.355276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:01.356235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:01.356580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:01.356912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] PATH_INFO: `/v3/users/18acadfb64174598a0da8e4f201704bc` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:01.358582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18acadfb64174598a0da8e4f201704bc)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:01.361827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:01.432052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18acadfb64174598a0da8e4f201704bc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.441725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18acadfb64174598a0da8e4f201704bc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.442624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18acadfb64174598a0da8e4f201704bc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.457535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18acadfb64174598a0da8e4f201704bc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.458325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18acadfb64174598a0da8e4f201704bc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.472284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1086/2150] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:01 2022] DELETE /identity/v3/users/18acadfb64174598a0da8e4f201704bc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:01.602469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:01.605382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:01.625628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52eecf-f939-447e-939d-e0255d92a072 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:01.635747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:01.734157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:01.735102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:01.735464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:01.735740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] PATH_INFO: `/v3/projects/075c55672a194d419bb114ce5bfb4e0d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:01.738108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075c55672a194d419bb114ce5bfb4e0d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:01.741315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:01.749135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:01.750616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:01.770848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6047cf-4b66-4774-89af-af222dd5dd92 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:01.793515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:01.794663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:01.795539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:01.796161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] PATH_INFO: `/v3/projects/90d0c8429a754800bbf952e05ba4b393` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:01.797587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d0c8429a754800bbf952e05ba4b393)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:01.802059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:01.823190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075c55672a194d419bb114ce5bfb4e0d', 'request_id': 'req-5adecd50-5314-41ba-8000-05a7a54ae4f8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.830758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075c55672a194d419bb114ce5bfb4e0d', 'request_id': 'req-5adecd50-5314-41ba-8000-05a7a54ae4f8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.837473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075c55672a194d419bb114ce5bfb4e0d', 'request_id': 'req-5adecd50-5314-41ba-8000-05a7a54ae4f8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.853672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] The token cache is being invalidate because project 075c55672a194d419bb114ce5bfb4e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:01.853876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.854678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.856319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1065/2151] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:01 2022] DELETE /identity/v3/projects/075c55672a194d419bb114ce5bfb4e0d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:01.860396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:01.863128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:01.871187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d0c8429a754800bbf952e05ba4b393', 'request_id': 'req-968a3740-5ffd-4930-8d0d-51131bb36d90'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.878367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d0c8429a754800bbf952e05ba4b393', 'request_id': 'req-968a3740-5ffd-4930-8d0d-51131bb36d90'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.892566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d0c8429a754800bbf952e05ba4b393', 'request_id': 'req-968a3740-5ffd-4930-8d0d-51131bb36d90'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.908679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] The token cache is being invalidate because project 90d0c8429a754800bbf952e05ba4b393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:01.908968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.909800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-968a3740-5ffd-4930-8d0d-51131bb36d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:01.914463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1087/2152] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:01 2022] DELETE /identity/v3/projects/90d0c8429a754800bbf952e05ba4b393 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:01.918392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc30d6f-3048-487d-a81a-6b64ff4de4cf None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:01.918877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc30d6f-3048-487d-a81a-6b64ff4de4cf None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:01.919157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc30d6f-3048-487d-a81a-6b64ff4de4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:01.930066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:01.943619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4dc30d6f-3048-487d-a81a-6b64ff4de4cf None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:01.945298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4dc30d6f-3048-487d-a81a-6b64ff4de4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:01.952202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adecd50-5314-41ba-8000-05a7a54ae4f8 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.025169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:02.048255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1088/2153] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:02.050924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.055552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.055910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.056207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] PATH_INFO: `/v3/users/fd4a5a41e68a41eab038e72c0254ade5` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.057535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4a5a41e68a41eab038e72c0254ade5)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.058398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.058899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.059541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.059647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.127354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:02.128592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.144883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1089/2154] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:02.151938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.167466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4a5a41e68a41eab038e72c0254ade5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.168319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4a5a41e68a41eab038e72c0254ade5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.171091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a221d74-f2c2-4b55-aad3-7dff495032e8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.181877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.182599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.182888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.183163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.184269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:02.184762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.185352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4a5a41e68a41eab038e72c0254ade5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.186431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.197368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4a5a41e68a41eab038e72c0254ade5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.198299 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4a5a41e68a41eab038e72c0254ade5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.202085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1090/2155] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:02.206574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.208904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.215431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1066/2156] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:01 2022] DELETE /identity/v3/users/fd4a5a41e68a41eab038e72c0254ade5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:02.224556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529afa-1f51-409f-ae9c-1d62f04e36b1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.346120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.347008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.348076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.348436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.349333 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.351722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.390767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1091/2157] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:02.391396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.421419 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250847c-c0fe-4d7b-b851-919b67c9c6cb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.466897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.468662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.496810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.497547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.497835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.498114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.499213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.501307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.532585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1067/2158] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:02.537593 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.549899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de75841-5516-488f-bb3a-fb6f5e337db9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.555340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.556383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.556669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.556950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.557692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.559414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.572057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1068/2159] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:02.576560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.588816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4954f-b652-4d2d-86c3-b07486e8c3a9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.596408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.597074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.597361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.597639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.598560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.600172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.609010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48520d35-e2af-4f6a-bcf0-dfe20c809747 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.612652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1069/2160] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:02.616406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.617766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.618661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.618947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.619221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] PATH_INFO: `/v3/projects/6420e518e3324af7b2e4f70426b9dc37` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.620422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6420e518e3324af7b2e4f70426b9dc37)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.622546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.627811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4224-80e4-4fa5-a359-069d984c95f2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.633526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.634358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:02.634534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.634906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.635252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] PATH_INFO: `/v3/projects/cdf47dedfddb4a9ab1e1599f9a3ec235/users/8d292a0b6d2c45f9b0945cd2238d714b/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.655108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf47dedfddb4a9ab1e1599f9a3ec235, user_id=8d292a0b6d2c45f9b0945cd2238d714b, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.659713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22839db-58fe-4675-8333-a85e12dc7722 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.675610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6420e518e3324af7b2e4f70426b9dc37', 'request_id': 'req-eb36a494-6458-413a-931d-e31347a22350'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.678211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1070/2161] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:02 2022] PUT /identity/v3/projects/cdf47dedfddb4a9ab1e1599f9a3ec235/users/8d292a0b6d2c45f9b0945cd2238d714b/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:02.686200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6420e518e3324af7b2e4f70426b9dc37', 'request_id': 'req-eb36a494-6458-413a-931d-e31347a22350'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.690958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c246-daee-444b-9900-0e5ad7a5a8a8 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.696966 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6420e518e3324af7b2e4f70426b9dc37', 'request_id': 'req-eb36a494-6458-413a-931d-e31347a22350'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.697202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c246-daee-444b-9900-0e5ad7a5a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.697442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882c246-daee-444b-9900-0e5ad7a5a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.711248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] The token cache is being invalidate because project 6420e518e3324af7b2e4f70426b9dc37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:02.711522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.713842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.715462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1092/2162] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:02 2022] DELETE /identity/v3/projects/6420e518e3324af7b2e4f70426b9dc37 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:02.716280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.717909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.727759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:02.737579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3882c246-daee-444b-9900-0e5ad7a5a8a8 None None] MFA Rules not processed for user `8d292a0b6d2c45f9b0945cd2238d714b`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:02.739887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3882c246-daee-444b-9900-0e5ad7a5a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.802538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36a494-6458-413a-931d-e31347a22350 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.836935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1071/2163] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:02.845349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.845690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.845985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.871349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:02.871349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.882584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:02.883445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:02.883725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:02.884001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] PATH_INFO: `/v3/projects/d201fe06026244d5b176b68db54ee38f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:02.885182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d201fe06026244d5b176b68db54ee38f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:02.887659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:02.946379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1072/2164] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:02.946379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:02.952115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:02.966451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce35c-7147-4c24-8926-cef647f3c21c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:02.972306 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:02.973662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd201fe06026244d5b176b68db54ee38f', 'request_id': 'req-ad9a0bde-9322-428d-aa93-f7f701546c6a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.981766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd201fe06026244d5b176b68db54ee38f', 'request_id': 'req-ad9a0bde-9322-428d-aa93-f7f701546c6a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:02.993310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd201fe06026244d5b176b68db54ee38f', 'request_id': 'req-ad9a0bde-9322-428d-aa93-f7f701546c6a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:03.006939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] The token cache is being invalidate because project d201fe06026244d5b176b68db54ee38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:03.007258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:03.007971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad9a0bde-9322-428d-aa93-f7f701546c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:03.013723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1093/2165] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:02 2022] DELETE /identity/v3/projects/d201fe06026244d5b176b68db54ee38f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:03.013723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7a2b-cd9c-47e1-93d8-32691faefa9b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.013723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7a2b-cd9c-47e1-93d8-32691faefa9b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.013723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd7a2b-cd9c-47e1-93d8-32691faefa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.037523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a5bd7a2b-cd9c-47e1-93d8-32691faefa9b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:03.038792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a5bd7a2b-cd9c-47e1-93d8-32691faefa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.042914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.043813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.044100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.044372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.045390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.046915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1094/2166] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:03.048484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.051370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.051612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.051895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.086699 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:03.088073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.092396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1073/2167] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:02 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.092396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.116316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:03.151820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1095/2168] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:03.152642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.153911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.162540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5a7a-be92-4d15-9d4c-b88845b99a5a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.165788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8939b-4108-46bc-8746-ab2786addeae None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.169865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.170184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.170560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.170860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.170981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.171139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.171231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.171412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.171868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:03.172087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.172193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.174443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e033c6-c833-4785-a136-142c1e161e0b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.174578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.186363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1074/2169] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:03.187770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.188061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.188340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.200785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1096/2170] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/users => generated 300 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.201635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.211597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:03.212245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06eef3-5824-4a9a-b7de-b886446d4602 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.212820 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.217004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.217648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.217930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.218204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.218923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.220858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.221103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1075/2171] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:03.224431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.231215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689ecfa-96d3-47cf-a84b-6b9aa0e3cc73 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.236018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.237163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.237450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.237808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.239137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.240991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.250110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1097/2172] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.251116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.257382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1076/2173] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.258376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.268001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4368b5e-3265-43eb-8703-f90fa6a0f960 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.323164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a141b1b-2afc-4535-9905-909d4ea8512b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.328107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.328760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.329041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.329341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.330104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:03.330701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.332037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.340092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.340712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.340998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.341271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.341960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.343997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.347892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1098/2174] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:03.347892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.357608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9495be-9713-4252-b2e3-f131eb2117c1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.363443 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.364293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.365024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.365024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] PATH_INFO: `/v3/projects/699589e25f9146ac8855699272a8c930/users/981cd86accfb438b874022b66991e460/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.374861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1077/2175] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.375536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.384857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699589e25f9146ac8855699272a8c930, user_id=981cd86accfb438b874022b66991e460, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.387253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3238db-37b5-41f5-bdd6-6f1d3f892de3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.387580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.393458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.394272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.394646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.394872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.395814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.397256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:03.398570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.402922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1099/2176] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:03 2022] PUT /identity/v3/projects/699589e25f9146ac8855699272a8c930/users/981cd86accfb438b874022b66991e460/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:03.407103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.440310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1078/2177] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.440310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.483101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d5513-1cc5-42c1-85a3-14216bb77020 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.525431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3701df-e75e-4649-9bb3-83c0641d2e26 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.534486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.535422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.535711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.536286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.537029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.538867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.556300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1079/2178] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.556300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.558564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.559269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.559555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.559846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.560630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.561110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:03.562804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.566203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143482f-9c00-4ea6-8bd0-861baf2c6bb1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.573012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.573777 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.574300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.574667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.575362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1100/2179] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.575494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.577396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.578818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.589132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382fba3-0599-4271-b072-ed4b7a44cb51 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.594732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.596471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.596749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.597031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] PATH_INFO: `/v3/projects/699589e25f9146ac8855699272a8c930/users/981cd86accfb438b874022b66991e460/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.599301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699589e25f9146ac8855699272a8c930, user_id=981cd86accfb438b874022b66991e460, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.601989 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.602707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1080/2180] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:03.603817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.613960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1101/2181] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:03 2022] PUT /identity/v3/projects/699589e25f9146ac8855699272a8c930/users/981cd86accfb438b874022b66991e460/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:03.614864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.615756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3f048-d011-490a-a943-711519dc1e03 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.683619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ad34c-180a-4e0e-abcd-1c3e5ea32011 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.685470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.686205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.686581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.686945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.687673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.688318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.688979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.689293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a887b0-b8e0-4a42-ad6f-f7757584a60f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.689399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.689569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.690345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.691937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.699730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1081/2182] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.700929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f077-5244-4c61-83e3-1d912a269f6a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.701256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f077-5244-4c61-83e3-1d912a269f6a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.701498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f077-5244-4c61-83e3-1d912a269f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.706692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1102/2183] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.706692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.714760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039dcb9-0ca2-4c4b-a0a1-4b4e02fa3cb7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.719441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.720324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.721155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.721442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] PATH_INFO: `/v3/projects/7d815dbc5bb54b0eaa67978e1dfe4aff/users/2edc8f3e1906419688485551bb625a8e/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.732873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d984f077-5244-4c61-83e3-1d912a269f6a None None] MFA Rules not processed for user `981cd86accfb438b874022b66991e460`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:03.734087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d984f077-5244-4c61-83e3-1d912a269f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.735998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d815dbc5bb54b0eaa67978e1dfe4aff, user_id=2edc8f3e1906419688485551bb625a8e, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.739263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.753239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1103/2184] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:03 2022] PUT /identity/v3/projects/7d815dbc5bb54b0eaa67978e1dfe4aff/users/2edc8f3e1906419688485551bb625a8e/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:03.762325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.762325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:03.808298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1082/2185] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:03.809631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.809916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.810197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.835046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6e7a5-663d-420e-a410-070ac8856480 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.839070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] MFA Rules not processed for user `2edc8f3e1906419688485551bb625a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:03.839922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.840348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.840591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.840883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.841161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.841912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.843664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:03.855788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1104/2186] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:03.856523 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.858365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:03.921473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:03.932109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1083/2187] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:03.932955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:03.946155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc1fc-c778-43e6-ba33-565196335265 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.946478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd6ded-eff7-4597-b5d2-06fe9fed4d16 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:03.950178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.950909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.951255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.951358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:03.951649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.952150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:03.952431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:03.952704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] PATH_INFO: `/v3/projects/6a4beb48dfc14400b1872c2930ea1e11/users/0021d731c3344934b33b5ed2b35284be/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:03.977990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4beb48dfc14400b1872c2930ea1e11, user_id=0021d731c3344934b33b5ed2b35284be, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:03.980830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.000053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1084/2188] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:03 2022] PUT /identity/v3/projects/6a4beb48dfc14400b1872c2930ea1e11/users/0021d731c3344934b33b5ed2b35284be/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:04.000972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.005490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.029030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.031617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5351fe8-b604-4faa-955e-b347620c4dda service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.048635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1105/2189] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:04.050808 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.051554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.051894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.082151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:04.084082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.093766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d03de8-10c0-489d-82b1-8ec13f814cf4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.104962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.105631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.106001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.106174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.168381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.169861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.172310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:04.179249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1085/2190] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:04.180034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.188439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1106/2191] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:04.199453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.200846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.247779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d835a-2589-4a93-be5d-bcd23ab63b6f service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.252353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.253214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.253491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.253768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] PATH_INFO: `/v3/users/8d292a0b6d2c45f9b0945cd2238d714b` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.254978 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d292a0b6d2c45f9b0945cd2238d714b)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.256893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.257493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5101a-e5ad-469d-aece-175a10a32479 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.261160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.261830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.262112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.262418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.275422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.276974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.288987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1107/2192] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:04.289842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.302870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2e030-ff25-469a-9eaf-8ffe3d25e09e service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.304540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:04.309122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.310554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.310888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.311172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] PATH_INFO: `/v3/users/2edc8f3e1906419688485551bb625a8e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.312582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2edc8f3e1906419688485551bb625a8e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.315419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.325849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d292a0b6d2c45f9b0945cd2238d714b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.326901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d292a0b6d2c45f9b0945cd2238d714b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.341890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d292a0b6d2c45f9b0945cd2238d714b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.353899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d292a0b6d2c45f9b0945cd2238d714b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.354615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4ba667e8-c277-4734-bf48-0552c5cd7f1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d292a0b6d2c45f9b0945cd2238d714b'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.373184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1086/2193] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:04 2022] DELETE /identity/v3/users/8d292a0b6d2c45f9b0945cd2238d714b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:04.378486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2edc8f3e1906419688485551bb625a8e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.379887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.380237 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.380565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.392881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2edc8f3e1906419688485551bb625a8e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.394032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2edc8f3e1906419688485551bb625a8e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.409965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:04.411378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.411795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2edc8f3e1906419688485551bb625a8e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.412915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-876404bf-da36-4ab1-ad8e-46e0498f4b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2edc8f3e1906419688485551bb625a8e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:04.436736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1108/2194] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:04 2022] DELETE /identity/v3/users/2edc8f3e1906419688485551bb625a8e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:04.442059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.442447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.442804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.463890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:04.469997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.511686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1087/2195] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:04.526457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.528127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.558069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1109/2196] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:04.569555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:04.572286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:04.586269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:04.604602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7a3d5-c56e-4e62-b637-5cb16b35b3c9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.668283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8194e-72fc-48ec-a5be-d1ab25fd611a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:04.673142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.674704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.674818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.675044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.693195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:04.694156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:04.694914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:04.695505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:04.709623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.712029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.715224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:04.716691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:04.726360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1088/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:04.735889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1110/2198] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:05.420293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.422366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.432889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.435790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.446492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97728-8ce0-4877-88b0-0d096442c01c service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:05.456408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:05.457485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.457667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.457897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] PATH_INFO: `/v3/projects/7d815dbc5bb54b0eaa67978e1dfe4aff` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.459742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d815dbc5bb54b0eaa67978e1dfe4aff)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:05.462092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:05.464642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:05.492467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747e685-93f7-4618-a53a-af5575b4c80a service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:05.509984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:05.513034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.513405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.513834 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] PATH_INFO: `/v3/projects/cdf47dedfddb4a9ab1e1599f9a3ec235` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.515900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf47dedfddb4a9ab1e1599f9a3ec235)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:05.523014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:05.543129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d815dbc5bb54b0eaa67978e1dfe4aff', 'request_id': 'req-c9774d01-3860-41c9-8452-3cf2a75302d0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.552366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d815dbc5bb54b0eaa67978e1dfe4aff', 'request_id': 'req-c9774d01-3860-41c9-8452-3cf2a75302d0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.559903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d815dbc5bb54b0eaa67978e1dfe4aff', 'request_id': 'req-c9774d01-3860-41c9-8452-3cf2a75302d0'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.581006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] The token cache is being invalidate because project 7d815dbc5bb54b0eaa67978e1dfe4aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:05.581333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.581968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c9774d01-3860-41c9-8452-3cf2a75302d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.584096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1089/2199] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:05 2022] DELETE /identity/v3/projects/7d815dbc5bb54b0eaa67978e1dfe4aff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:05.589220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07591-8d98-4d2d-8275-ab2232e20e42 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.589502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07591-8d98-4d2d-8275-ab2232e20e42 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.589824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07591-8d98-4d2d-8275-ab2232e20e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.605554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf47dedfddb4a9ab1e1599f9a3ec235', 'request_id': 'req-28297781-bebe-4387-959f-4510dc1f183f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.619448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf47dedfddb4a9ab1e1599f9a3ec235', 'request_id': 'req-28297781-bebe-4387-959f-4510dc1f183f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.630783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-47a07591-8d98-4d2d-8275-ab2232e20e42 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:05.633258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-47a07591-8d98-4d2d-8275-ab2232e20e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.634681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf47dedfddb4a9ab1e1599f9a3ec235', 'request_id': 'req-28297781-bebe-4387-959f-4510dc1f183f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.657822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] The token cache is being invalidate because project cdf47dedfddb4a9ab1e1599f9a3ec235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:05.658172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.659326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.664993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1111/2200] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:05 2022] DELETE /identity/v3/projects/cdf47dedfddb4a9ab1e1599f9a3ec235 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:05.666623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.667865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.723361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1090/2201] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:05.733945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.734259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.734671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.763107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:05.764339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.788334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1091/2202] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:05.788556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28297781-bebe-4387-959f-4510dc1f183f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:05.789664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.794550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:05.796024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.796356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.796635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] PATH_INFO: `/v3/users/981cd86accfb438b874022b66991e460` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.798099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=981cd86accfb438b874022b66991e460)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:05.800904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:05.801571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28612f5-f312-42dd-af34-d9224f7060dd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:05.807826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:05.808633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.809395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.809755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.811030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:05.811557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:05.814688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:05.832121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1092/2203] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:05.837451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.851505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:05.855565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d32f2-b3c5-473b-b28c-c9aebddb6190 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:05.864490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:05.867089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:05.867725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:05.867911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '981cd86accfb438b874022b66991e460'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.868450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:05.869368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:05.873442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:05.880340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '981cd86accfb438b874022b66991e460'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.881212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '981cd86accfb438b874022b66991e460'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.905638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '981cd86accfb438b874022b66991e460'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.906735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '981cd86accfb438b874022b66991e460'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:05.918996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1093/2204] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:05 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:05.924770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:05.927672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:05.929879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1112/2205] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:05 2022] DELETE /identity/v3/users/981cd86accfb438b874022b66991e460 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:06.029520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:06.059875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35c10-138d-45dd-a5d9-05bd5c7ea0ff admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.208842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.213403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.214549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.214981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.219084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.222940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:06.251206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1094/2206] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:05 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:06.255780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:06.272316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135870ae-f649-4cc8-b60f-86bd7b78a2af admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.289475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.290466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.290614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.290965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.291773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.293757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:06.305913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1113/2207] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:06.310311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:06.324085 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631f666-0927-4c64-9874-20c1b2617787 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.332348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.333674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.333950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.334228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.335019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.339466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c15ab-026e-4634-9291-01fb636feec9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:06.353465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1095/2208] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:06.354058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:06.365434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfe191-913d-4f54-a59e-c0a1584ad059 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.385067 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.387636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.387927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.388211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] PATH_INFO: `/v3/projects/8afe8de6967949d695d4fee060327ba0/users/1a9f8a6306f0440eacce63b287484002/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.431466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afe8de6967949d695d4fee060327ba0, user_id=1a9f8a6306f0440eacce63b287484002, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.436374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:06.459903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1114/2209] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:06 2022] PUT /identity/v3/projects/8afe8de6967949d695d4fee060327ba0/users/1a9f8a6306f0440eacce63b287484002/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:06.471644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.471925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.472207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.478665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:06.480297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.520995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] MFA Rules not processed for user `1a9f8a6306f0440eacce63b287484002`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:06.522557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.641132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d644b4f-a972-4637-8f04-f6fdc16dcd3e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.662017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.662936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.663219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.664172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] PATH_INFO: `/v3/projects/699589e25f9146ac8855699272a8c930` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.667144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699589e25f9146ac8855699272a8c930)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.669128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:06.687818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1096/2210] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:06.703471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:06.706381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.762809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699589e25f9146ac8855699272a8c930', 'request_id': 'req-8492cbb1-f722-47b2-9bd8-264d79641172'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:06.772479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699589e25f9146ac8855699272a8c930', 'request_id': 'req-8492cbb1-f722-47b2-9bd8-264d79641172'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:06.790212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699589e25f9146ac8855699272a8c930', 'request_id': 'req-8492cbb1-f722-47b2-9bd8-264d79641172'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:06.811162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b716-4aa6-408e-9e08-0b9a202d161a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:06.812082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] The token cache is being invalidate because project 699589e25f9146ac8855699272a8c930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:06.812619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:06.813656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8492cbb1-f722-47b2-9bd8-264d79641172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:06.818868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1115/2211] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:06 2022] DELETE /identity/v3/projects/699589e25f9146ac8855699272a8c930 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:06.824198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b2b53-b64c-4708-b8a8-7f262302953a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.824768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b2b53-b64c-4708-b8a8-7f262302953a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.825269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b2b53-b64c-4708-b8a8-7f262302953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.834677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:06.837706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:06.858070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ad1b2b53-b64c-4708-b8a8-7f262302953a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:06.860604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ad1b2b53-b64c-4708-b8a8-7f262302953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.895227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:06.895897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.896480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.896768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.898654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.948299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1116/2212] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:06.962176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:06.966204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:06.966204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:06.992910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:06.994793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:06.997023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:06.998004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.006454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1117/2213] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:07.008769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1097/2214] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:07.011980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.020662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2905b1-bc2c-482c-a555-5629136d9a01 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.025607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.026369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.026734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.027081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.027881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:07.028258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.029941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.044192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1118/2215] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 17:54:07.048988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.061200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afb522-b6f3-4afd-b567-8c98a28d0ba8 service glance] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.067524 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.068273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.068752 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.069094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.069858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.072087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.131586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.131881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1098/2216] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:07 2022] POST /identity/v3/projects => generated 393 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:07.145185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e6d58-383f-4520-8306-c5bfe43ab110 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.267997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.269122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.269415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.269691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.270432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.272831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.323227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1119/2217] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:07 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:07.327606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.346626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9c42a-ed93-437e-b642-b1e43a9718f3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.353105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.354079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.354404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.354689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.355429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.357054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.373475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1099/2218] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:07.376083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.389254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3189a6a-35a9-4a66-af4f-4c5d34087915 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.402578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.404764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.405101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.406528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.408733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.413910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd6ad2-eda7-4890-adf3-a82daf3b3e8e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.445670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1120/2219] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:07.448511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.473093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814dd0-86f4-459e-9470-7671aaf49d21 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.495328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.496395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.496770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.497158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] PATH_INFO: `/v3/projects/1c7c4bff2b484cd2b1663a1c5e114d21/users/4175ba17b210492f99e579206ff847ff/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.530808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7c4bff2b484cd2b1663a1c5e114d21, user_id=4175ba17b210492f99e579206ff847ff, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.538566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.569996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1100/2220] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:07 2022] PUT /identity/v3/projects/1c7c4bff2b484cd2b1663a1c5e114d21/users/4175ba17b210492f99e579206ff847ff/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:07.583956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.584768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.591017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.664287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] MFA Rules not processed for user `4175ba17b210492f99e579206ff847ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:07.665600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:07.802321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:07.827077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1121/2221] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:07.828311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:07.925701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e44df-39e9-49f7-9b0c-ccdf87f36fc9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:07.931854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:07.932677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:07.933089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:07.933570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:07.934449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:07.936617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:07.999608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1101/2222] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:07 2022] POST /identity/v3/projects => generated 393 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:08.004064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.037625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76f43-7fc3-4fd9-9f56-420ddfe9be93 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:08.250750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:08.251363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.252187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.252464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.253167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:08.255175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:08.296491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1122/2223] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:08 2022] POST /identity/v3/users => generated 303 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:08.300554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.318591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:08.334089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a9c8-9eed-4502-afad-ee47f0017b17 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:08.349628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:08.350313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.350620 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.350899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.352544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:08.358485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:08.410590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1102/2224] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:08.412207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.428234 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f8f39-d1c6-458c-83b5-8497df6b0adc admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:08.447212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:08.450307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.450639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.450928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] PATH_INFO: `/v3/projects/7724ce6dcc5e44559d5f22805f19e59b/users/103a45dc54d74301a5eb363c08db4df7/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.502921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7724ce6dcc5e44559d5f22805f19e59b, user_id=103a45dc54d74301a5eb363c08db4df7, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:08.505553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:08.532960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1123/2225] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:08 2022] PUT /identity/v3/projects/7724ce6dcc5e44559d5f22805f19e59b/users/103a45dc54d74301a5eb363c08db4df7/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:08.537720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.549368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f61be4-ccde-470c-8962-eba05ca96bbd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:08.679156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:08.680329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.680842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.681340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.682279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:08.684153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e87bf-11e3-4100-af30-23a3d6604e43 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:08.707704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1103/2226] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:08.712223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.725300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c433d3-bf0b-4e7a-a635-d5bbf95fefb5 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:08.747468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:08.748500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.748856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.749200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] PATH_INFO: `/v3/projects/7724ce6dcc5e44559d5f22805f19e59b/users/103a45dc54d74301a5eb363c08db4df7/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.751358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7724ce6dcc5e44559d5f22805f19e59b, user_id=103a45dc54d74301a5eb363c08db4df7, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:08.755826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:08.769512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1124/2227] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:08 2022] PUT /identity/v3/projects/7724ce6dcc5e44559d5f22805f19e59b/users/103a45dc54d74301a5eb363c08db4df7/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:08.784457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:08.785213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:08.785678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:08.851451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] MFA Rules not processed for user `103a45dc54d74301a5eb363c08db4df7`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:08.852653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:08.975720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1104/2228] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:08.990613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:08.994189 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:09.094223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63373592-3999-4528-9c9d-52ec8d124e90 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:09.099857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:09.100671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:09.100958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:09.101246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:09.124843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:09.126353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6f356-2a34-41ed-be9b-eafbacec48a0 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:09.148766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1125/2229] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:09.232348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:09.233876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:09.271642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2612cf1-ee83-4445-aee6-b07d5826778c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:09.286521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:09.287547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:09.288128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:09.288617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:09.479807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:09.481619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:09.512453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1105/2230] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:10.605387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:10.605952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:10.605952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:10.649779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] MFA Rules not processed for user `0021d731c3344934b33b5ed2b35284be`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:10.651222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:10.707659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:10.756815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1126/2231] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:10.758544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:10.759515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:10.789695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:10.874280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b9db-8f33-4c2b-92b1-3df9f79d01c0 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:10.883227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:10.884000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:10.884353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:10.884698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:10.885506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:10.887611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:10.935993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1106/2232] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:10 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:10.937213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:10.951019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae417a8-d6cc-4dfc-86a3-b46a6301c014 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:11.035088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:11.035771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:11.036264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:11.036556 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:11.037269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:11.040292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:11.062567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1127/2233] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:10 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:11.067185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:11.078816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073255-87ff-4e02-9e59-3856974cfe0e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:11.085061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:11.085825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:11.086591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:11.086591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:11.087316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:11.089007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:11.105078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1107/2234] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:11.108962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:11.120644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17800a-99fb-49f9-ac8e-57bcce1da15c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:11.126102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:11.127790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:11.128118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:11.128402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] PATH_INFO: `/v3/projects/881379e27aa0423dbe219c76acb6649a/users/d7709f1e82dd4ccb8b04e8756c31e8c0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:11.148286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=881379e27aa0423dbe219c76acb6649a, user_id=d7709f1e82dd4ccb8b04e8756c31e8c0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:11.150993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:11.181993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1128/2235] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:11 2022] PUT /identity/v3/projects/881379e27aa0423dbe219c76acb6649a/users/d7709f1e82dd4ccb8b04e8756c31e8c0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:11.185260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:11.200543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0cb4-9ae0-47cc-875e-4e780ddde374 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:11.323264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:11.324246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:11.324531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:11.324811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:11.325649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:11.327581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:11.346921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:11.350538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1108/2236] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:11.361356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978c892-3a4f-412a-9700-f84e04b8ef5b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:11.374154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:11.376948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:11.376948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:11.376948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] PATH_INFO: `/v3/projects/881379e27aa0423dbe219c76acb6649a/users/d7709f1e82dd4ccb8b04e8756c31e8c0/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:11.378521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=881379e27aa0423dbe219c76acb6649a, user_id=d7709f1e82dd4ccb8b04e8756c31e8c0, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:11.381275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:11.401333 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1129/2237] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:11 2022] PUT /identity/v3/projects/881379e27aa0423dbe219c76acb6649a/users/d7709f1e82dd4ccb8b04e8756c31e8c0/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:12.174458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:12.179847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:12.201232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f37d3-db16-42f7-bbaa-2f29747f46e1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:12.292239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:12.293118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:12.293400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:12.293671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] PATH_INFO: `/v3/users/52a3c04395c64202b10451c270bac9c5` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:12.304132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a3c04395c64202b10451c270bac9c5)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:12.307092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:12.342390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:12.397233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a3c04395c64202b10451c270bac9c5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:12.400528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a3c04395c64202b10451c270bac9c5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:12.428473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a3c04395c64202b10451c270bac9c5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:12.445482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a3c04395c64202b10451c270bac9c5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:12.446175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d5df2e8c-00ea-4341-8ccf-224754a86a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a3c04395c64202b10451c270bac9c5'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:12.463954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1109/2238] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:12 2022] DELETE /identity/v3/users/52a3c04395c64202b10451c270bac9c5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:12.840198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:12.841995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:12.870013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759e5a2-bb5a-44e0-849f-834ebc166f9c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:13.013322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:13.014354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.014706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.015051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] PATH_INFO: `/v3/projects/66e3ea2346c743ef96f525d34c031848` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.019884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e3ea2346c743ef96f525d34c031848)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:13.021927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:13.117903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:13.142241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e3ea2346c743ef96f525d34c031848', 'request_id': 'req-1f50e021-5013-444e-9bee-4dea0b6264fe'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:13.149943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e3ea2346c743ef96f525d34c031848', 'request_id': 'req-1f50e021-5013-444e-9bee-4dea0b6264fe'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:13.162707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e3ea2346c743ef96f525d34c031848', 'request_id': 'req-1f50e021-5013-444e-9bee-4dea0b6264fe'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:13.195714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] The token cache is being invalidate because project 66e3ea2346c743ef96f525d34c031848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:13.196020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:13.206435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-1f50e021-5013-444e-9bee-4dea0b6264fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:13.213143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1130/2239] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:12 2022] DELETE /identity/v3/projects/66e3ea2346c743ef96f525d34c031848 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:13.218366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.218869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.218966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.254421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:13.255805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:13.342369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1110/2240] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:13.348513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.349014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.349409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.396365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:13.398545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:13.412977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1131/2241] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:13.417479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:13.427953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77b05f-b390-478d-b5db-c4dfb52b4873 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:13.451115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:13.451952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.452354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.452695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.453427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:13.453848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:13.455533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:13.501668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:13.502182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1111/2242] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:13.513789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc89f1-97d9-4cda-ab0e-f201e47abb6a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:13.542368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:13.543084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.543369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.544172 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.547583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:13.557456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:13.613224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1132/2243] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:13 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:13.617779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:13.647013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beccec-572b-4ed4-b99b-e24a1209d780 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:13.833862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:13.834763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:13.835121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:13.835469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:13.837600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:13.841666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:13.924844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1112/2244] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:13 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:13.926456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:13.979498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60323b-0a61-46b7-a107-fb51988a6d36 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:14.025135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:14.026133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:14.027019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:14.027503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:14.028494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:14.032022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:14.054970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1133/2245] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:14.058893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:14.075122 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748b5f8-ffe0-4532-8627-6c91a6eed106 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:14.082184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:14.083292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:14.083648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:14.083995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:14.085543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:14.087163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646aa941-bd9f-41d0-8547-e563f17c145d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:14.109219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1113/2246] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:14.114320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:14.145218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88331517-19c4-4de5-b683-4da22ae09632 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:14.155279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:14.156345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:14.156743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:14.157107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] PATH_INFO: `/v3/projects/1dc70bdf33a44324a55e8a894a7340ee/users/e031883951b8486581e198cf6c9705b8/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:14.205221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc70bdf33a44324a55e8a894a7340ee, user_id=e031883951b8486581e198cf6c9705b8, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:14.208264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:14.252779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1134/2247] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:14 2022] PUT /identity/v3/projects/1dc70bdf33a44324a55e8a894a7340ee/users/e031883951b8486581e198cf6c9705b8/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:14.258079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:14.258803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:14.258967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:14.300913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:14.313033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:14.314869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:14.453366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1114/2248] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:14.462459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:14.464611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:14.560243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0df334-5658-4caf-8054-878b29f17ef9 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:14.565711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:14.566082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:14.566391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:14.566972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:14.583892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:14.585359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa81bc-d04b-40ad-9cf6-a5e0561cbc2d service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:14.599459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1135/2249] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:15.138704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:15.138704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:15.162396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995193f7-6c49-436e-a290-01e86f8ad59b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:15.173570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:15.174958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:15.175391 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:15.175858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] PATH_INFO: `/v3/users/4175ba17b210492f99e579206ff847ff` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:15.184251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4175ba17b210492f99e579206ff847ff)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:15.190918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:15.281840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4175ba17b210492f99e579206ff847ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:15.283853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4175ba17b210492f99e579206ff847ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:15.297149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4175ba17b210492f99e579206ff847ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:15.339250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4175ba17b210492f99e579206ff847ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:15.341266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4175ba17b210492f99e579206ff847ff'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:15.370960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1115/2250] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:15 2022] DELETE /identity/v3/users/4175ba17b210492f99e579206ff847ff => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:15.373358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:15.374247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:15.374897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:15.404365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:15.417260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:15.549257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:15.579284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1136/2251] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:15.589444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:15.591322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:15.712731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3a0ea-5046-4de7-908b-11518de1aad5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:15.717699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:15.718914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:15.719496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:15.720675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:15.743717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:15.745107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:15.786106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1116/2252] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:16.230367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:16.232208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:16.253249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82244946-c569-4423-924d-7c4a8a05d8f8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:16.272998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:16.274919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:16.275359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:16.276415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] PATH_INFO: `/v3/projects/1c7c4bff2b484cd2b1663a1c5e114d21` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:16.278612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7c4bff2b484cd2b1663a1c5e114d21)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:16.281291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:16.369342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7c4bff2b484cd2b1663a1c5e114d21', 'request_id': 'req-9c400eaf-20f0-4f5c-810a-2228ce5577a7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.378649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7c4bff2b484cd2b1663a1c5e114d21', 'request_id': 'req-9c400eaf-20f0-4f5c-810a-2228ce5577a7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.394670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7c4bff2b484cd2b1663a1c5e114d21', 'request_id': 'req-9c400eaf-20f0-4f5c-810a-2228ce5577a7'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.418527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] The token cache is being invalidate because project 1c7c4bff2b484cd2b1663a1c5e114d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:16.420015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.421364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.425305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1137/2253] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:16 2022] DELETE /identity/v3/projects/1c7c4bff2b484cd2b1663a1c5e114d21 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:16.431499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:16.468967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e30b1-decf-40ac-b7ee-a4f4cec80ad8 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:16.515016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:16.608834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:16.611658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:16.612161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:16.612717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] PATH_INFO: `/v3/users/103a45dc54d74301a5eb363c08db4df7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:16.616650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=103a45dc54d74301a5eb363c08db4df7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:16.618901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:16.730850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103a45dc54d74301a5eb363c08db4df7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.731730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103a45dc54d74301a5eb363c08db4df7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.750353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103a45dc54d74301a5eb363c08db4df7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.771181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103a45dc54d74301a5eb363c08db4df7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.772149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-121cb8fe-e8e7-4cd7-86d6-135436e3c0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103a45dc54d74301a5eb363c08db4df7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:16.793310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1117/2254] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:16 2022] DELETE /identity/v3/users/103a45dc54d74301a5eb363c08db4df7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:17.581793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:17.584499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:17.636148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c400eaf-20f0-4f5c-810a-2228ce5577a7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:17.801893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:17.803094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:17.804248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:17.804707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] PATH_INFO: `/v3/projects/7724ce6dcc5e44559d5f22805f19e59b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:17.806177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7724ce6dcc5e44559d5f22805f19e59b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:17.808906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:17.899762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7724ce6dcc5e44559d5f22805f19e59b', 'request_id': 'req-4dd30419-fb1f-4c1e-93cb-46032c2c430a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:17.912526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7724ce6dcc5e44559d5f22805f19e59b', 'request_id': 'req-4dd30419-fb1f-4c1e-93cb-46032c2c430a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:17.925176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7724ce6dcc5e44559d5f22805f19e59b', 'request_id': 'req-4dd30419-fb1f-4c1e-93cb-46032c2c430a'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:17.941667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] The token cache is being invalidate because project 7724ce6dcc5e44559d5f22805f19e59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:17.941667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:17.942604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-4dd30419-fb1f-4c1e-93cb-46032c2c430a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:17.945423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1138/2255] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:17 2022] DELETE /identity/v3/projects/7724ce6dcc5e44559d5f22805f19e59b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:17.950466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:17.950968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:17.951045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.000360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:18.003342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:18.150199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1118/2256] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:18.162036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.162322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.162546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.195383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:18.196629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:18.211898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1139/2257] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:18.216670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.234249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c8d53-6836-430b-9e20-8c110a81e515 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.244566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.248803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.249314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.249423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.252242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:18.252581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.258017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a78926-2b0a-4566-b9c2-8caa5da0a28d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.288320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1119/2258] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:18.290002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.316301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0b30e-a901-4374-a990-4eaffda02856 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.331160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.331830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.332633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.332911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.333933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.341909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.351544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:18.354457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.354916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.355340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.396132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1140/2259] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:18 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:18.399958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.412695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] MFA Rules not processed for user `d7709f1e82dd4ccb8b04e8756c31e8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:18.413931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:18.476173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:18.578033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d3675-a354-495c-897b-dc2b070c8eb3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.584941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.585770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.586145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.586667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.587516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.590155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.590537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1120/2260] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:18.605230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.607695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:18.650111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1141/2261] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:18 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:18.654158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.672725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc9d6-e1b8-48ff-948f-f3bb2aa7a17f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.689405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.690934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.692750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.693138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.695451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.700797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.728209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1142/2262] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:18.729482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.737446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506707f-553f-4b12-8242-a5c3584698ff None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.743302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.744102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.744553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.744845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.746917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:18.747765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9c56-cb3a-421a-a9fd-4a0725545f9d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.754125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.755047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.755977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.756565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.757766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.760197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.774976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1143/2263] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:18.780842 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:18.800564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74fe95-e2c4-4672-9588-5436094bce6e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:18.808955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:18.810208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.810652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.811036 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] PATH_INFO: `/v3/projects/0fe08226f17847bdb71e39154dd08545/users/eb6ff53c36314328b3e946307c5ef669/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.834827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe08226f17847bdb71e39154dd08545, user_id=eb6ff53c36314328b3e946307c5ef669, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.842086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:18.844482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35d10-3d71-4b19-816d-187aa99270a9 service glance] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.855750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1121/2264] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:54:18.865270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5ca09-ca68-4f94-ba9f-d4a843f22385 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:18.921756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1144/2265] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:18 2022] PUT /identity/v3/projects/0fe08226f17847bdb71e39154dd08545/users/eb6ff53c36314328b3e946307c5ef669/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:18.933072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:18.933072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:18.933339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:18.978601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] MFA Rules not processed for user `eb6ff53c36314328b3e946307c5ef669`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:18.979960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:19.097290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.097592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.098511 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.103019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:19.113603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1122/2266] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:19.118783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.119545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:19.120831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:19.213021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51f46-6a66-42ed-8d0d-052e5f384529 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.234649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1145/2267] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:54:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:19.235186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:19.236405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.236690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.236980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.237668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:19.239697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:19.304557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1123/2268] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:19 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:19.308877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.309028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.311148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:19.326183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd81fe6-425a-49f1-bc54-1d11c2a7b3b0 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.448688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:19.542809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:19.542809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.542809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.542809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.543195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:19.543195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:19.549319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12d2c-a477-449c-8c9d-4e948a7340f7 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.577762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:19.578464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.578754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.579363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.633045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1146/2269] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:19 2022] POST /identity/v3/users => generated 299 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:19.641206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.680578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c47337-57ff-467e-8368-5b3e67fd4c65 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.688629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:19.689552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.689978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.690470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.691356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:19.693136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:19.727470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1147/2270] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:19.735055 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.750162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:19.753473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:19.770569 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3139e-5d4d-4044-9001-ac8b801bf242 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.778205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1124/2271] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 17:54:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 17:54:19.798366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:19.799373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:19.799742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:19.799926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] PATH_INFO: `/v3/projects/8ddc2a2713144b89a6596e7287aed51d/users/a1c160986c7748769f093420d0595d60/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:19.859235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ddc2a2713144b89a6596e7287aed51d, user_id=a1c160986c7748769f093420d0595d60, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:19.862209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:19.893331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1148/2272] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:19 2022] PUT /identity/v3/projects/8ddc2a2713144b89a6596e7287aed51d/users/a1c160986c7748769f093420d0595d60/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:19.897262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:19.932898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642e3c9-92e6-4c2f-8429-a97dc9f93fa0 service swift] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:19.994809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:20.129830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:20.130716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:20.131111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:20.131590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:20.132428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:20.135198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0ac55-1a4c-4e84-8402-8c3203cf2f9c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:20.158040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1125/2273] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:20.162653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:20.175959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26e21-140f-47cd-b9f1-4b688611f33f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:20.193041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:20.196130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:20.199454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:20.200165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] PATH_INFO: `/v3/projects/8ddc2a2713144b89a6596e7287aed51d/users/a1c160986c7748769f093420d0595d60/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:20.211489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ddc2a2713144b89a6596e7287aed51d, user_id=a1c160986c7748769f093420d0595d60, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:20.218544 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:20.302821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1149/2274] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:20 2022] PUT /identity/v3/projects/8ddc2a2713144b89a6596e7287aed51d/users/a1c160986c7748769f093420d0595d60/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:20.316519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:20.316910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:20.317289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:20.385278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] MFA Rules not processed for user `a1c160986c7748769f093420d0595d60`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:20.386669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:20.602718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1126/2275] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:20.613129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:20.616681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:20.773639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce482a08-e1ec-44f7-bebc-210cacce0c3e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:20.781369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:20.784387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:20.785034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:20.785483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:20.812619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:20.814419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb031f-b6fb-4c2d-9c16-03af1869c6f2 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:20.828686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1150/2276] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:22.240650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:22.373590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5a845-37f3-4d2d-b796-8259e95221c1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:22.401826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:22.402717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:22.403010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:22.403289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] PATH_INFO: `/v3/users/eb6ff53c36314328b3e946307c5ef669` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:22.404585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6ff53c36314328b3e946307c5ef669)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:22.406580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:22.523322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6ff53c36314328b3e946307c5ef669'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:22.526594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6ff53c36314328b3e946307c5ef669'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:22.532537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:22.556765 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6ff53c36314328b3e946307c5ef669'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:22.583554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6ff53c36314328b3e946307c5ef669'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:22.588354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6ff53c36314328b3e946307c5ef669'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:22.634497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1127/2277] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:22 2022] DELETE /identity/v3/users/eb6ff53c36314328b3e946307c5ef669 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:22.637867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9fc9c-6156-4f46-8825-630a7457bb65 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:22.638936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9fc9c-6156-4f46-8825-630a7457bb65 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:22.639435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9fc9c-6156-4f46-8825-630a7457bb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:22.665362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-aca9fc9c-6156-4f46-8825-630a7457bb65 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:22.666872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-aca9fc9c-6156-4f46-8825-630a7457bb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:22.754165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1151/2278] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:22.777143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:22.778800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:22.907629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61016f5-8e0f-4441-b1dd-d2f4ba987882 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:22.913832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:22.914519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:22.914802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:22.915077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:22.949491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:22.951066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf655e-48a1-4196-b3b0-5adbce3f7c3b service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:22.980564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:22.981165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:22.981674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:22.996044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1128/2279] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:54:23.042865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] MFA Rules not processed for user `e031883951b8486581e198cf6c9705b8`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:23.044098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:23.050022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:23.174974 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1152/2280] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:23.186289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:23.186737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:23.187136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:23.229513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] MFA Rules not processed for user `e031883951b8486581e198cf6c9705b8`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:23.235008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:23.265318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1129/2281] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:23.282668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:23.284464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:23.407347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abf9ed-f563-45c9-8d72-ccc675f31463 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:23.442225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:23.443232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:23.443677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:23.444111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:23.464269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:23.467182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:23.534882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1153/2282] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:54:23.538559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:23.541609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:23.573353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6bbbe-0701-4efd-98fa-1f777aceadd9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:23.580285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:23.581770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:23.582821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:23.582821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] PATH_INFO: `/v3/projects/0fe08226f17847bdb71e39154dd08545` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:23.584035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe08226f17847bdb71e39154dd08545)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:23.587169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:23.677470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe08226f17847bdb71e39154dd08545', 'request_id': 'req-4292b372-3555-4188-9773-b92bde705068'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:23.685258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe08226f17847bdb71e39154dd08545', 'request_id': 'req-4292b372-3555-4188-9773-b92bde705068'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:23.697257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe08226f17847bdb71e39154dd08545', 'request_id': 'req-4292b372-3555-4188-9773-b92bde705068'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:23.720679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] The token cache is being invalidate because project 0fe08226f17847bdb71e39154dd08545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:23.720983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:23.722678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:23.734048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:23.750779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1130/2283] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:23 2022] DELETE /identity/v3/projects/0fe08226f17847bdb71e39154dd08545 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:23.759410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14b13-c4da-448e-b27d-e5c65d367883 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:23.915168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:23.916450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:23.918019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:23.918645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] PATH_INFO: `/v3/users/a1c160986c7748769f093420d0595d60` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:23.920292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c160986c7748769f093420d0595d60)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:23.924614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:24.035164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:24.062624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c160986c7748769f093420d0595d60'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.084535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c160986c7748769f093420d0595d60'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.086046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c160986c7748769f093420d0595d60'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.118865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c160986c7748769f093420d0595d60'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.119662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-bb287c66-8f21-43c9-bf78-f476b53f3fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c160986c7748769f093420d0595d60'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.159321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1154/2284] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:23 2022] DELETE /identity/v3/users/a1c160986c7748769f093420d0595d60 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:24.513283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:24.515416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:24.539958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b372-3555-4188-9773-b92bde705068 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:24.632360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:24.657485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:24.658407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:24.658698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:24.658975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] PATH_INFO: `/v3/projects/8ddc2a2713144b89a6596e7287aed51d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:24.661272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ddc2a2713144b89a6596e7287aed51d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:24.670726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:24.726762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cea7d-523e-4ca5-be5b-11c3711ea346 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:24.727297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cea7d-523e-4ca5-be5b-11c3711ea346 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:24.727297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cea7d-523e-4ca5-be5b-11c3711ea346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:24.805263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-df8cea7d-523e-4ca5-be5b-11c3711ea346 None None] MFA Rules not processed for user `e031883951b8486581e198cf6c9705b8`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:24.807444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-df8cea7d-523e-4ca5-be5b-11c3711ea346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:24.811347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ddc2a2713144b89a6596e7287aed51d', 'request_id': 'req-00e3dc80-27de-4eaa-afba-42c1c8d74161'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.839772 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ddc2a2713144b89a6596e7287aed51d', 'request_id': 'req-00e3dc80-27de-4eaa-afba-42c1c8d74161'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.866083 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ddc2a2713144b89a6596e7287aed51d', 'request_id': 'req-00e3dc80-27de-4eaa-afba-42c1c8d74161'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.920166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] The token cache is being invalidate because project 8ddc2a2713144b89a6596e7287aed51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:24.921079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.924267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-00e3dc80-27de-4eaa-afba-42c1c8d74161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:24.926724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1131/2285] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:24 2022] DELETE /identity/v3/projects/8ddc2a2713144b89a6596e7287aed51d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:24.935059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:24.935638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:24.935931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:24.949097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1155/2286] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:24.995223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:24.996569 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:25.183431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1132/2287] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:25.184969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.185367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.185740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.225532 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:25.226960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:25.242958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1156/2288] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:25.247588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.260600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0372-b866-4c27-9dfb-0c052f5ba6c8 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.275041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.275715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.276002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.276282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.285090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:25.285451 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.287304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.318786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.347468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1133/2289] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:25.351270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8954-9dfc-4bb8-9bf6-5130cbada1e8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.362244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.364488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.366817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.367226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.368174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.370471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.451962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1157/2290] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:25 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:25.457622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.471509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def2b9-26a7-4f3a-bf44-a1cae9272a50 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.593603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.594503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.594891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.595284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.596071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.598111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.606790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:25.637264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.644013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1134/2291] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:25 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:25.652809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254d7ff-e90a-48dd-9b90-e597fe7247de admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.665690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.666394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.666728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.667012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.667751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.670033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.699500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1158/2292] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:25.704166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.721834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d9251-e949-45f3-8f7b-253495076e1d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.737770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.738708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.740069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.740069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.740269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.747238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380f71-bee7-4bfe-9439-4e988a484e1f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.781584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1135/2293] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:25.786560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:25.797388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c695b-21d4-4fd4-bbb7-a712f91869f0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:25.803415 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:25.804319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.804599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.804879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] PATH_INFO: `/v3/projects/38d5abbad7ff4625b96f3ac7c7768e5e/users/15f9866c8cba40dc869b223b6d8041b4/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.824263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d5abbad7ff4625b96f3ac7c7768e5e, user_id=15f9866c8cba40dc869b223b6d8041b4, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:25.827791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:25.854452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1159/2294] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:25 2022] PUT /identity/v3/projects/38d5abbad7ff4625b96f3ac7c7768e5e/users/15f9866c8cba40dc869b223b6d8041b4/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:25.861271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223de2f-e03a-4ddc-983d-38e3f036685f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:25.862919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223de2f-e03a-4ddc-983d-38e3f036685f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:25.863372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223de2f-e03a-4ddc-983d-38e3f036685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:25.903742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2223de2f-e03a-4ddc-983d-38e3f036685f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:25.905192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2223de2f-e03a-4ddc-983d-38e3f036685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:26.199372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1136/2295] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:26.205825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:26.207606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:26.487694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f337-2410-4ae1-b3aa-d21673183e09 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:26.512462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:26.513429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:26.513429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:26.513731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:26.551230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:26.553892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:26.555885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:26.572738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1160/2296] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:29.050286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:29.051206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:29.051660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:29.114203 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] MFA Rules not processed for user `15f9866c8cba40dc869b223b6d8041b4`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:29.115885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:29.324536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1137/2297] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:29.329551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:29.363384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31de723-889a-426c-a855-8d9e008493f9 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:29.382396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:29.383449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:29.383970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:29.384462 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:29.386136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:29.388892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:29.470433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1161/2298] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:29 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:29.476643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:29.556284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d3d3d-adcc-4939-aa94-8d4b9d82d758 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:29.721714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:29.722807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:29.723345 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:29.723844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:29.724923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:29.728316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:29.751269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:29.761834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1138/2299] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:29 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 17:54:29.763898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:29.779638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83741-c2fa-4a8d-9283-82932ad37096 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:29.794942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:29.795968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:29.796455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:29.796939 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:29.798716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:29.800581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:29.841607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1162/2300] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:29.846468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:29.862185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370671-6d83-4b21-a8b1-541b7e7e2541 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:29.884216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:29.887925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:29.888311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:29.888520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] PATH_INFO: `/v3/projects/173a39e124ce457eac3f917d6c322587/users/aa1d0bf884054023aaa47cc7e47dfd2d/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:29.931889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=173a39e124ce457eac3f917d6c322587, user_id=aa1d0bf884054023aaa47cc7e47dfd2d, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:29.942062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:30.001110 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1139/2301] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:29 2022] PUT /identity/v3/projects/173a39e124ce457eac3f917d6c322587/users/aa1d0bf884054023aaa47cc7e47dfd2d/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:30.004793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:30.016608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd42892-5413-4601-b421-3663d4d587be admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:30.143952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:30.144629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:30.145489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:30.145674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:30.147093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:30.149455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfd884-eb35-43eb-ad9e-5dc9e322e9b7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:30.174304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1163/2302] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:30.181484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:30.196198 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1cd7b-aacf-4b06-be87-7341df5bab7c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:30.206217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:30.207235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:30.207591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:30.207941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] PATH_INFO: `/v3/projects/173a39e124ce457eac3f917d6c322587/users/aa1d0bf884054023aaa47cc7e47dfd2d/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:30.210088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173a39e124ce457eac3f917d6c322587, user_id=aa1d0bf884054023aaa47cc7e47dfd2d, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:30.213219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:30.226740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1140/2303] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:30 2022] PUT /identity/v3/projects/173a39e124ce457eac3f917d6c322587/users/aa1d0bf884054023aaa47cc7e47dfd2d/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:31.698139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:31.698730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:31.698952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:31.737802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] MFA Rules not processed for user `aa1d0bf884054023aaa47cc7e47dfd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:31.739635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:31.913624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1164/2304] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:31.924254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:31.926301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:32.056949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975e3b9-ade4-45a0-b18e-d1dad917f781 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:32.071472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:32.072145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:32.072809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:32.073282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:32.198460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:32.200057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d788798-a1fc-4ce4-9945-7575281b7390 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:32.224460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1141/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:34.786164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:34.787613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:34.804637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0be656-4ac6-478b-afd3-ac0f9d97a081 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:34.808353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:34.810079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:34.810079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:34.810233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:34.822737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:34.824794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ff1ae-c2ec-4b09-876a-b1a02c2f334e service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:34.840781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1165/2306] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:36.748462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:36.749055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:36.749055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:36.767278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] MFA Rules not processed for user `0021d731c3344934b33b5ed2b35284be`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:36.768460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:36.824700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1142/2307] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:44.833374 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:46.818247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:52.859858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:52.860436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:52.860436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:52.879369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:52.880858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:52.957499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1166/2308] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:54:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 17:54:52.979306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:52.980910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:53.107797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9047a-0845-4a10-8f67-04b4fe2517d0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:53.115108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:53.115686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:53.116130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:53.116522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:53.130461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:53.133414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ed12d-85f9-4c6e-9c76-597fb90dcc5c service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:53.144106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1143/2309] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 17:54:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 17:54:56.984519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:56.986696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.057081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60ab5-0752-45bd-a80d-f4b64aa365a7 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:57.061587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:57.062516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.062803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.063077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] PATH_INFO: `/v3/users/de89581b05ea4cada13321ea59b54402` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.064423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de89581b05ea4cada13321ea59b54402)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:57.066375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:57.114964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de89581b05ea4cada13321ea59b54402'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.122707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de89581b05ea4cada13321ea59b54402'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.123405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de89581b05ea4cada13321ea59b54402'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.136251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de89581b05ea4cada13321ea59b54402'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.137271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de89581b05ea4cada13321ea59b54402'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.150670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1167/2310] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:56 2022] DELETE /identity/v3/users/de89581b05ea4cada13321ea59b54402 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:57.155238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.155687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.155795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.175601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:57.177042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.191491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:57.193118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.211752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:57.263826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ecbaf-9b6f-45bb-9f98-5e7e719874c4 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:57.267716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1144/2311] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:57.270575 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:57.271609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.272064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.272420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] PATH_INFO: `/v3/users/1a9f8a6306f0440eacce63b287484002` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.273737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9f8a6306f0440eacce63b287484002)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:57.274053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:57.275514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.275919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:57.341000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a72af-dcfc-4585-93f8-65d8fff39ba1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:57.352144 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9f8a6306f0440eacce63b287484002'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.357597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:57.358712 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.359188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.362735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.372869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9f8a6306f0440eacce63b287484002'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.373589 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9f8a6306f0440eacce63b287484002'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.392595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:57.394205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a4fe5-67b6-4733-9389-bad2021614c8 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:57.397836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9f8a6306f0440eacce63b287484002'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.398959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9f8a6306f0440eacce63b287484002'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:57.408166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1145/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:54:57.433304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1168/2313] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:54:57 2022] DELETE /identity/v3/users/1a9f8a6306f0440eacce63b287484002 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:57.437746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.438342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.438819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.475086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:57.476635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.559811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:57.657671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1146/2314] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:57.669482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:57.671554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:57.825179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbdfa6-0a8a-4785-ad38-527fd8f5f987 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:57.840119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:57.840795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:57.841078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:57.841353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:57.875232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:57.879625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:57.909664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1169/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:54:57.973568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:57.975367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:58.004895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cbaa6-70d5-4d2a-8f30-7818353356df None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:58.010939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:58.014112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.014589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.016221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] PATH_INFO: `/v3/projects/537911eafd204734a304a41bf8fde679` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.021481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=537911eafd204734a304a41bf8fde679)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:58.024803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:58.127597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537911eafd204734a304a41bf8fde679', 'request_id': 'req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.148029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537911eafd204734a304a41bf8fde679', 'request_id': 'req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.163841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537911eafd204734a304a41bf8fde679', 'request_id': 'req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.192811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] The token cache is being invalidate because project 537911eafd204734a304a41bf8fde679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:58.195759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.197109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-6e62675f-4c89-4a61-8015-cd6cc12b3d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.202039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1147/2316] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:57 2022] DELETE /identity/v3/projects/537911eafd204734a304a41bf8fde679 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:58.579859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:58.581253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:58.597061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929fb5-5748-4e75-bfe6-b0f17b2ab92b service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:58.652571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:58.669662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:58.670559 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.670900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.671179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] PATH_INFO: `/v3/projects/8afe8de6967949d695d4fee060327ba0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.672369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afe8de6967949d695d4fee060327ba0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:58.674600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:58.729495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afe8de6967949d695d4fee060327ba0', 'request_id': 'req-578455d2-6800-4e3b-a0ca-e908fc46c9d1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.735711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afe8de6967949d695d4fee060327ba0', 'request_id': 'req-578455d2-6800-4e3b-a0ca-e908fc46c9d1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.743203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afe8de6967949d695d4fee060327ba0', 'request_id': 'req-578455d2-6800-4e3b-a0ca-e908fc46c9d1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.754896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] The token cache is being invalidate because project 8afe8de6967949d695d4fee060327ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:54:58.755186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.755845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-578455d2-6800-4e3b-a0ca-e908fc46c9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:54:58.757396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1170/2317] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:54:58 2022] DELETE /identity/v3/projects/8afe8de6967949d695d4fee060327ba0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:54:58.762156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.762480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.762775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.784192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:58.785440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:58.855864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1148/2318] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:58.864554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.864838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.865122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.887243 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:58.888505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:58.910826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1171/2319] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:58.912060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:58.922387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371f12-7031-41e8-ab48-d9928c6c8df3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:58.927627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:58.928368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.928716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.929056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.929850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:54:58.930247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:58.931947 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:58.948297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1149/2320] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:54:58.950071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:58.966486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba813c6b-4e9c-4ce9-90c0-e4eabefd582b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:58.973211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:58.973878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:58.974166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:58.974482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:58.975177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:58.977257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.023534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1172/2321] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:54:58 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:59.023858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:59.036846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239365a-f445-4ea3-a8c5-15297ea75ef6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:59.098491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:54:59.119624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:59.120397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.121332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.122090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.123004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:59.125226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.155059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1150/2322] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:54:59 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:54:59.156316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:59.173517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d2e-c986-41c7-9792-c2cc147e526c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:59.180587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:59.181708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.182190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.182685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.183792 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:59.185459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.206128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1173/2323] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 17:54:59.209873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:59.222232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2baadc-7ee3-4d7c-8147-c3dfcf38d8b4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:59.228837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:59.229527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.229815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.230103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.230885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:59.232515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c28941-393e-4f19-9959-edc542082256 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.244865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1151/2324] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:54:59.249030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:59.261983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075330a-05f4-4db1-9b07-3bdcfbe2115a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:59.267369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:59.268316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.268962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.269250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] PATH_INFO: `/v3/projects/885b0b9f03c14c7a9b318cc567850bf3/users/89104f082e0242eb883d62e945188829/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.286257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885b0b9f03c14c7a9b318cc567850bf3, user_id=89104f082e0242eb883d62e945188829, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:59.289000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.306694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1174/2325] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:54:59 2022] PUT /identity/v3/projects/885b0b9f03c14c7a9b318cc567850bf3/users/89104f082e0242eb883d62e945188829/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:54:59.317475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.317843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.318228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.347375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] MFA Rules not processed for user `89104f082e0242eb883d62e945188829`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:54:59.348663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:59.414750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1152/2326] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:54:59.426271 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:54:59.427951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:54:59.485073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48278-b8e9-4a30-9d54-fc5d910305f4 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:54:59.489500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:54:59.490043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:54:59.490350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:54:59.490652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:54:59.499183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:54:59.500612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848208bb-9f49-406c-b54f-53a2ccb15ec5 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:54:59.510139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1175/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:54:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:55:09.411831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:55:09.506079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:55:56.629378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:55:56.630845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:55:56.788079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a761d68-4c5d-4ced-b77a-d4c18d2618dd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:55:56.793216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:55:56.793854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:55:56.794131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:55:56.794430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:55:56.796184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:55:56.808635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:55:56.810209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd03c6-0233-4074-ae44-646989007903 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:55:56.820827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1153/2328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:55:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:55:59.232367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d374-c16d-42cc-8d5a-5ce7d371c3ef None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:55:59.232963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d374-c16d-42cc-8d5a-5ce7d371c3ef None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:55:59.232963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d374-c16d-42cc-8d5a-5ce7d371c3ef None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:55:59.234314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1176/2329] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jun 30 17:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 17:55:59.239881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:55:59.240326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:55:59.240459 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:55:59.295724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:55:59.297121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:55:59.306564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1154/2330] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:55:59.332944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d374-c16d-42cc-8d5a-5ce7d371c3ef None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:55:59.347920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d374-c16d-42cc-8d5a-5ce7d371c3ef None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:55:59.352964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:55:59.353599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:55:59.353885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:55:59.354163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:55:59.362935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:55:59.364541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:55:59.374304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1177/2331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:55:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:56:07.698839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:07.700380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:07.758368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:56:07.821963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2542a-8d67-4519-9fea-da26828de345 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:07.826482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:07.827355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:07.827637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:07.827913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] PATH_INFO: `/v3/users/e031883951b8486581e198cf6c9705b8` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:07.829086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e031883951b8486581e198cf6c9705b8)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:07.831587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:07.877679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e031883951b8486581e198cf6c9705b8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:07.878244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e031883951b8486581e198cf6c9705b8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:07.890397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e031883951b8486581e198cf6c9705b8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:07.898701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e031883951b8486581e198cf6c9705b8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:07.899042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-50eb0292-d07c-4cc8-8c98-72bb172f917d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e031883951b8486581e198cf6c9705b8'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:07.911450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1155/2332] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 17:56:07 2022] DELETE /identity/v3/users/e031883951b8486581e198cf6c9705b8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:56:08.103427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.115624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d6ed-27f3-4481-be9f-70335a73ccdd service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.172973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.173847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.174128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.174424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] PATH_INFO: `/v3/projects/1dc70bdf33a44324a55e8a894a7340ee` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.175584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc70bdf33a44324a55e8a894a7340ee)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.177471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.222358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc70bdf33a44324a55e8a894a7340ee', 'request_id': 'req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:08.227706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc70bdf33a44324a55e8a894a7340ee', 'request_id': 'req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:08.234684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc70bdf33a44324a55e8a894a7340ee', 'request_id': 'req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:08.249751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] The token cache is being invalidate because project 1dc70bdf33a44324a55e8a894a7340ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 17:56:08.250183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:08.251065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5385f3e0-b954-4789-8e3a-1d4bbf896fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 17:56:08.253912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1178/2333] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 17:56:08 2022] DELETE /identity/v3/projects/1dc70bdf33a44324a55e8a894a7340ee => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 17:56:08.264393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.264732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.265058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.312538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:56:08.313762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:08.371996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1156/2334] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:56:08.380321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.380567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.380843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.398677 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:56:08.399876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:08.407747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1179/2335] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:56:08.411866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.421618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906042fa-fd4a-4aec-998c-7a96ec51d1cf None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.426645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.427306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.427586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.427902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.428638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 17:56:08.428939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.430543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.438819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:56:08.443343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1157/2336] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 17:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 17:56:08.447670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.456585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783462a-0d1b-4f48-b7e6-3139c1eba133 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.461252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.461883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.462163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.462463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.463167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.465075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.489772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1180/2337] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 17:56:08 2022] POST /identity/v3/projects => generated 375 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:56:08.493871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.504372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87633a-1e3a-4bcc-b8c7-b7588c0e2ffa admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.560037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.560690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.560975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.561252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.561927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.563883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.584258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1158/2338] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 17:56:08 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 17:56:08.588553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.598851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d67b2d-b5be-49c6-aba3-c28821803a16 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.603433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.604092 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.604379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.604654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.605399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.607042 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.617872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1181/2339] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:56:08.621771 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.632013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3b5a5-fcaa-4ef0-aeeb-059809d1999e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.636440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.637113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.637392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.637662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.638413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.640479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a567946-066a-4536-811f-852ad9a6fc26 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.650948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1159/2340] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 17:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 17:56:08.655229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.665602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083aac7-fa96-434a-8128-537223cd8237 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.670142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.671043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.671323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.671594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] PATH_INFO: `/v3/projects/221481e35182443995ea17136383f6f4/users/de0ef30352514ef9bb4559473e096ea6/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.686255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221481e35182443995ea17136383f6f4, user_id=de0ef30352514ef9bb4559473e096ea6, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.688965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.701609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1182/2341] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 17:56:08 2022] PUT /identity/v3/projects/221481e35182443995ea17136383f6f4/users/de0ef30352514ef9bb4559473e096ea6/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 17:56:08.706523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf610-8662-4dbb-bd6d-d60c5a434526 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.706994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf610-8662-4dbb-bd6d-d60c5a434526 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.707104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcf610-8662-4dbb-bd6d-d60c5a434526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.725631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-69fcf610-8662-4dbb-bd6d-d60c5a434526 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:56:08.726850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-69fcf610-8662-4dbb-bd6d-d60c5a434526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:08.785824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1160/2342] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:56:08.795563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:08.797236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:08.818663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:56:08.882884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26d0d7-6279-4085-beb5-578e828183b7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:08.886533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:08.887168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:08.887449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:08.887723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:08.896930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:08.898370 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:08.909290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1183/2343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:56:12.944042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c14219-988b-44ea-a2da-e5269a8743ef None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:12.944566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c14219-988b-44ea-a2da-e5269a8743ef None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:12.944566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c14219-988b-44ea-a2da-e5269a8743ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:12.975383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e4c14219-988b-44ea-a2da-e5269a8743ef None None] MFA Rules not processed for user `de0ef30352514ef9bb4559473e096ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:56:12.976608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e4c14219-988b-44ea-a2da-e5269a8743ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:13.058233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1161/2344] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:56:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:56:13.072510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:56:13.074081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:13.182573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d4a0-6bc3-4036-8632-429311df8e12 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:56:13.188140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:56:13.188788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:13.189607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:13.189888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:13.198704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:56:13.200340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:56:13.212605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1184/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:56:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:56:15.885959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:56:15.886502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:56:15.886773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:56:15.905047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] MFA Rules not processed for user `de0ef30352514ef9bb4559473e096ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 17:56:15.906351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:56:15.913844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1162/2346] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 17:56:23.207192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=75 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:56:25.900022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:57:56.425791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:57:56.427358 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:57:56.504381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f8d69-653b-454c-8638-a23972061e8d service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:57:56.507960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:57:56.508755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:57:56.509037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:57:56.509309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:57:56.511045 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:57:56.568013 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:57:56.569429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:57:56.578357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1185/2347] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:57:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:57:58.316726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:57:58.326454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f1fc0-b348-4694-ab3c-1d8f9663dd9a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:57:58.329930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:57:58.330580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:57:58.330864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:57:58.331134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:57:58.332865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:57:58.389887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:57:58.391348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:57:58.400071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1163/2348] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:58:06.574224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=57 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:58:08.396056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=28 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:58:53.387105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:58:53.388517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:58:53.399242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38e8ee-3e6a-45e7-8963-f8354a428d0d service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:58:53.402937 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:58:53.403600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:58:53.403885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:58:53.404161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:58:53.406005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:58:53.472251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:58:53.473676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:58:53.482673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1186/2349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:58:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:58:54.673396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:58:54.682754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb8262-bbb9-479c-b0c6-e24d0fc9f454 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:58:54.686912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:58:54.687545 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:58:54.687824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:58:54.688106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:58:54.689846 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:58:54.699456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:58:54.700901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:58:54.708854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1164/2350] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:59:03.478367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=28 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:59:04.705204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:59:18.858632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:59:18.867151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0907920-5ac8-46e6-8048-50002b54ece2 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:59:18.870755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:59:18.871437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:59:18.871722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:59:18.872000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:59:18.873766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:59:18.930537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:59:18.931996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:59:18.941015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1187/2351] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:59:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 17:59:28.936836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=28 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:59:31.759455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:59:31.768680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03571d6-b431-408c-9c72-902540ab6803 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:59:31.772133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:59:31.772959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:59:31.773066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:59:31.773278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:59:31.775021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:59:31.833304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:59:31.834782 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:59:31.843986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1165/2352] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:59:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 17:59:41.839519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=28 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 17:59:58.516308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 17:59:58.524812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286c01c-0200-4fd1-a19d-d84bce4eb613 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 17:59:58.528568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 17:59:58.529323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 17:59:58.529469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 17:59:58.529744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 17:59:58.531624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 17:59:58.593134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 17:59:58.594515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 17:59:58.603831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1188/2353] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 17:59:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:00:08.599248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=28 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:01:13.791462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:01:13.800130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f76dd-09b2-4e7d-9652-6b2608247ea0 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:01:13.803729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:01:13.804358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:01:13.804642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:01:13.804913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:01:13.813543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:01:13.815003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:01:13.823727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1166/2354] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:01:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:01:23.819718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:01:54.924110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:01:54.925489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:01:54.994774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2b27a-8f5c-4445-ae5a-556f8f3cb211 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:01:54.999769 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:01:55.000417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:01:55.000702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:01:55.000978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:01:55.002811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:01:55.014786 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:01:55.016381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:01:55.026114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1189/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:01:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:02:05.021908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=32 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:02:55.399508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:02:55.409121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6eb44-db71-4062-84f2-df920672dc6e service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:02:55.412625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:02:55.413247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:02:55.413526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:02:55.413845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:02:55.415597 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:02:55.429095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:02:55.430508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:02:55.441955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1167/2356] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:02:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:02:56.581310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:02:56.596503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e1a0-8307-41d2-b04e-b2d303db7687 service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:02:56.603062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:02:56.603743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:02:56.604025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:02:56.604299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:02:56.613716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:02:56.615341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:02:56.647001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1190/2357] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:02:56.704735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:02:56.706363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:02:56.717804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9778-4c48-4385-bf71-b7568c593b5b service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:02:56.721609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:02:56.722320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:02:56.722721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:02:56.723060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:02:56.724919 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:02:56.739285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:02:56.740946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:02:56.753159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1168/2358] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:02:57.451307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:02:57.453003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:02:57.468850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168c42b-3491-4e3a-b2b9-50ffe985e947 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:02:57.475210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:02:57.475925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:02:57.476207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:02:57.476478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:02:57.478291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:02:57.498426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:02:57.500873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a99962-6f23-46c8-8ea5-d6408d99187c service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:02:57.526366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1191/2359] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:02:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:02:58.408007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:02:58.420037 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba45b0-d42c-40af-b1fb-a5a7951c419a service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:02:58.427804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:02:58.428552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:02:58.428837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:02:58.429159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:02:58.446421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:02:58.447872 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:02:58.464519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1169/2360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:01.385327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762d52-726f-49b1-b760-db057a2e4541 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:01.385832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762d52-726f-49b1-b760-db057a2e4541 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:01.385965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762d52-726f-49b1-b760-db057a2e4541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:01.436581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-76762d52-726f-49b1-b760-db057a2e4541 None None] MFA Rules not processed for user `63422b2e34eb479f972a181fb37f3608`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:03:01.438077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-76762d52-726f-49b1-b760-db057a2e4541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:01.445195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1192/2361] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:03:01.921759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:01.932157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe435-47ce-4f56-b3da-e1ba81012831 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:01.936732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:01.937953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:01.937953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:01.938304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:01.940036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:01.950685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:01.952120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:01.967128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1170/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:03.196967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:03.197444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:03.197555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:03.244198 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] MFA Rules not processed for user `2ed5e98a29c44296aeb921af4f521042`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:03:03.245447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:03.261373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1193/2363] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:03:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:03:03.320121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:03.332751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b95bc8-1940-4983-ae40-22b8a64adf97 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:03.337971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:03.338668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:03.338975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:03.339817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:03.341742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:03.353293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:03.355553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:03.365695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1171/2364] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:11.492911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:11.523846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0111ce2-19cc-40c1-a0fb-099aca1ab670 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:11.539826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:11.540500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:11.540790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:11.541074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:11.543463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:11.557798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:11.559478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f1eeb-f900-4964-9f64-1d9a2dbced49 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:11.572071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1194/2365] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:11.841803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:11.857288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fa3f3-2a49-47a6-83c0-f19a22d400a0 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:11.862883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:11.863537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:11.863811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:11.864323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:11.866143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:11.881304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:11.883028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:11.894635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1172/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:21.567527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=55 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:03:21.889845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=36 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:03:51.617816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:51.618409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:51.618409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:51.636115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] MFA Rules not processed for user `de0ef30352514ef9bb4559473e096ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:03:51.637302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:51.644648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1195/2367] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:03:51.942730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:51.952901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd87041-4ef2-4e4e-86c8-f181836313da service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:51.957687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:51.958353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:51.958802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:51.959096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:51.960968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:03:51.972747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:51.974176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:51.982608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1173/2368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:52.712660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:52.723965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5084fc2-d9af-4166-b2f7-ba2cabf165a4 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:52.727795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:52.728437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:52.728723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:52.729001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:52.737890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:52.739388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:52.748704 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1196/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:03:54.681615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:03:54.693831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda521d-5ddf-418c-aea5-77067a504bbd service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:03:54.697713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:03:54.698379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:03:54.698670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:03:54.698945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:03:54.707928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:03:54.709370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:03:54.718044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1174/2370] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:03:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:04:02.744350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=9 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:04:04.714066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=12 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:04:18.309442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:04:18.318916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7afb4-e963-4bc0-ab3f-1bc7d25b0be0 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:04:18.322580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:04:18.323225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:04:18.323507 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:04:18.323780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:04:18.332696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:04:18.334114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:04:18.360270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1197/2371] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:04:28.357669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=13 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:04:31.067134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:04:31.076402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aab2a4-5d3c-4ffc-aac2-05da093e9802 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:04:31.079917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:04:31.080535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:04:31.080840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:04:31.081109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:04:31.089199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:04:31.090629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:04:31.118423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1175/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:04:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:04:41.115440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=13 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:04:57.890029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:04:57.901645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cd67e-34f7-4bc2-a115-4ff8f1ba2c30 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:04:57.905341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:04:57.905979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:04:57.906287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:04:57.906597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:04:57.915199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:04:57.916619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:04:57.925621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1198/2373] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:04:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:05:07.921427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=6 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:06:13.624878 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:06:13.642843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312fc7-b4ae-4839-b288-8851d11fc993 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:06:13.646483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:06:13.647104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:06:13.647387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:06:13.647657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:06:13.649478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:06:13.751022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:06:13.752469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:06:13.770930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1176/2374] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:06:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:06:23.768225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=56 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:06:58.941130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:06:59.002559 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4d04-ca25-4024-b7ff-73c563cedc90 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:06:59.007695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:06:59.008331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:06:59.008613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:06:59.008885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:06:59.017999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:06:59.019620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3e999-eccd-466c-b9f4-2e799dd9edf1 service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:06:59.029377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1199/2375] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:06:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:06:59.695360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:06:59.697067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:06:59.793265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a62fa3-e298-4425-b496-5d0e7001dd51 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:06:59.798486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:06:59.799353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:06:59.799638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:06:59.799912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] PATH_INFO: `/v3/users/89104f082e0242eb883d62e945188829` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:06:59.812819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89104f082e0242eb883d62e945188829)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:06:59.815338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:06:59.862306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89104f082e0242eb883d62e945188829'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:06:59.862982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89104f082e0242eb883d62e945188829'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:06:59.874486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89104f082e0242eb883d62e945188829'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:06:59.882135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89104f082e0242eb883d62e945188829'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:06:59.882800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89104f082e0242eb883d62e945188829'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:06:59.895184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1177/2376] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:06:59 2022] DELETE /identity/v3/users/89104f082e0242eb883d62e945188829 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:06:59.899715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:06:59.900069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:06:59.900413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:06:59.938904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:06:59.940190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:00.010101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1200/2377] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:07:00.020528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.021902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:00.065112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:00.101465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39902ea8-9cf8-4b67-b513-a1dabfc43175 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.105096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.105735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.106016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.106364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.115951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.117373 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf378908-c234-4174-a216-3607a749fac8 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.126818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1178/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:00.410154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.411610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:00.424996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e1a5e-6308-49a5-9e3d-b49d6a33c71c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.430218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.431123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.431412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.431741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] PATH_INFO: `/v3/projects/885b0b9f03c14c7a9b318cc567850bf3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.446064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885b0b9f03c14c7a9b318cc567850bf3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.448200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.503258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885b0b9f03c14c7a9b318cc567850bf3', 'request_id': 'req-fe7a795f-d9c6-4832-9253-087cee6cd646'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:00.509079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885b0b9f03c14c7a9b318cc567850bf3', 'request_id': 'req-fe7a795f-d9c6-4832-9253-087cee6cd646'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:00.517843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885b0b9f03c14c7a9b318cc567850bf3', 'request_id': 'req-fe7a795f-d9c6-4832-9253-087cee6cd646'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:00.532816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] The token cache is being invalidate because project 885b0b9f03c14c7a9b318cc567850bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:07:00.533102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:00.533760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-fe7a795f-d9c6-4832-9253-087cee6cd646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:00.535354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1201/2379] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:07:00 2022] DELETE /identity/v3/projects/885b0b9f03c14c7a9b318cc567850bf3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:07:00.541329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.541489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.541762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.592136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:00.594375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:00.657064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1179/2380] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:07:00.664997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.665286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.665561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.683525 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:00.684747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:00.693661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1202/2381] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:07:00.697966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.709708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4152a-0193-4669-aedb-7de3abf89fd9 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.714212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.714889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.715169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.715438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.716152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:07:00.716455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.718071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.733201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1180/2382] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:07:00.738158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.742717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:00.748892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3068bdd-dcad-4c11-9137-70a0bbefb780 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.754952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.755600 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.755882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.756161 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.756847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.758794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.795185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1203/2383] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:07:00 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:07:00.799969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.809999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897b8ed-9460-4dfe-90dc-8b766d41647c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.876400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.877076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.877356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.877635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.878324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.881490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.904907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1181/2384] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:07:00 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:07:00.906104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.916965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72244039-fc93-4f2b-a75b-19bb8a947d55 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.921518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.922167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.922486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.923250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.924082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.925837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.939394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1204/2385] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:07:00.942187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.952323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3dc95-6b36-4714-97df-aca30a4004b9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:00.957516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:00.958185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:00.958499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:00.958779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:00.959653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:00.961265 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bf21f-d5c0-44e7-a056-123ef2d5ba1c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:00.973282 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1182/2386] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:07:00.977290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:00.996799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdab31-3c87-4969-9dbf-4172449cbc02 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:01.002557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:01.004485 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:01.004577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:01.004810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] PATH_INFO: `/v3/projects/82c89f5009ad442985e92db48fc125df/users/ad1c8f646e0744a89e947061ee28255e/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:01.029071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c89f5009ad442985e92db48fc125df, user_id=ad1c8f646e0744a89e947061ee28255e, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:01.032046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:01.050376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1205/2387] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:07:00 2022] PUT /identity/v3/projects/82c89f5009ad442985e92db48fc125df/users/ad1c8f646e0744a89e947061ee28255e/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:07:01.066296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:01.067072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:01.067716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:01.078970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:01.100454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] MFA Rules not processed for user `ad1c8f646e0744a89e947061ee28255e`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:01.101667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:01.177251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1183/2388] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:07:01.188431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:01.190279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:01.250325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3ca6d-6450-42c4-94b5-a95a941aadb8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:01.254084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:01.254675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:01.254955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:01.255245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:01.263885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:01.265308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:01.270678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:01.272042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:01.274317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1206/2389] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:01.328329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cabbb-493d-4a3b-a368-44c612f56445 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:01.332504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:01.333226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:01.333576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:01.333917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:01.335836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:01.400824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:01.402259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:01.412448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1184/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:07:01.878924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:01.880483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:01.985169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77086c-b755-4e37-9cf4-eb61802d2219 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:01.994658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:01.995309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:01.995666 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:01.995888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:01.997801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:02.011509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:02.013445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:02.025231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1207/2391] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:02.780359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:02.782418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:02.794928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1b8e1-6951-46f8-9c9d-51f109095098 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:02.800932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:02.801829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:02.802113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:02.802435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] PATH_INFO: `/v3/users/15f9866c8cba40dc869b223b6d8041b4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:02.816871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f9866c8cba40dc869b223b6d8041b4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:02.818831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:02.848902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:02.867506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f9866c8cba40dc869b223b6d8041b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:02.868156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f9866c8cba40dc869b223b6d8041b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:02.880705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f9866c8cba40dc869b223b6d8041b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:02.890685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f9866c8cba40dc869b223b6d8041b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:02.891258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f9866c8cba40dc869b223b6d8041b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:02.909090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1185/2392] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:07:02 2022] DELETE /identity/v3/users/15f9866c8cba40dc869b223b6d8041b4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:07:03.085725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:03.096925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ab634-5206-4219-9495-0334794f92e7 service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:03.113134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:03.165350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:03.166272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.166592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.166872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] PATH_INFO: `/v3/projects/38d5abbad7ff4625b96f3ac7c7768e5e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.168074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d5abbad7ff4625b96f3ac7c7768e5e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:03.171002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:03.218581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d5abbad7ff4625b96f3ac7c7768e5e', 'request_id': 'req-ed1a205f-e15a-467a-9afd-fdae62035eb2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.223978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d5abbad7ff4625b96f3ac7c7768e5e', 'request_id': 'req-ed1a205f-e15a-467a-9afd-fdae62035eb2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.231137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d5abbad7ff4625b96f3ac7c7768e5e', 'request_id': 'req-ed1a205f-e15a-467a-9afd-fdae62035eb2'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.242703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] The token cache is being invalidate because project 38d5abbad7ff4625b96f3ac7c7768e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:07:03.242991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.243883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.245459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1208/2393] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:07:03 2022] DELETE /identity/v3/projects/38d5abbad7ff4625b96f3ac7c7768e5e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:07:03.249384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:03.251462 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:03.264130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ac8c-78c6-4986-b80c-246a3c1c140d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:03.319873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:03.320735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.321027 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.321290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] PATH_INFO: `/v3/users/aa1d0bf884054023aaa47cc7e47dfd2d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.334152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1d0bf884054023aaa47cc7e47dfd2d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:03.336394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:03.380812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1d0bf884054023aaa47cc7e47dfd2d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.381381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1d0bf884054023aaa47cc7e47dfd2d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.392844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1d0bf884054023aaa47cc7e47dfd2d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.400133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1d0bf884054023aaa47cc7e47dfd2d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.400780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f5072e23-9175-4626-81f0-9dc7616fffd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1d0bf884054023aaa47cc7e47dfd2d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.413883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1186/2394] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:07:03 2022] DELETE /identity/v3/users/aa1d0bf884054023aaa47cc7e47dfd2d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:07:03.597995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:03.599530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:03.614915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a205f-e15a-467a-9afd-fdae62035eb2 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:03.675039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:03.675846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.676130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.676406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] PATH_INFO: `/v3/projects/173a39e124ce457eac3f917d6c322587` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.687749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=173a39e124ce457eac3f917d6c322587)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:03.689712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:03.734270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:03.745934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173a39e124ce457eac3f917d6c322587', 'request_id': 'req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.752721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173a39e124ce457eac3f917d6c322587', 'request_id': 'req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.761030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173a39e124ce457eac3f917d6c322587', 'request_id': 'req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.782514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] The token cache is being invalidate because project 173a39e124ce457eac3f917d6c322587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:07:03.782802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.783466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-792b02d5-a84c-47d8-84c8-8bd9cfa5d37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:07:03.784981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1209/2395] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:07:03 2022] DELETE /identity/v3/projects/173a39e124ce457eac3f917d6c322587 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:07:03.790709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.791184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.791285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.809545 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:03.810780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:03.819940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:03.873594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1187/2396] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:07:03.881999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.882287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.882592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.899879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:03.901064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:03.908826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1210/2397] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:07:03.912767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:03.922208 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a71b5-cd0e-46bc-9472-e69bdb54d33b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:03.927374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:03.927955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.928241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.928515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.929260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:07:03.929566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:03.931207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:03.943532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1188/2398] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:07:03.947320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:03.962713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a957bea-5131-4550-aac0-7cf4c3dae78f None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:03.967923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:03.968574 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:03.968854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:03.969126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:03.969900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:03.972034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.003717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1211/2399] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:07:03 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:07:04.007616 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:04.017553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e10d23-d88b-4f57-a7ec-3ab5adc2e8d3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:04.072997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:04.073739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.074029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.074302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.075128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:04.076942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.101438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1189/2400] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:07:04 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:07:04.106958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:04.117736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d62de2-ff26-4714-a97c-a918e9701ee8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:04.122379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:04.123044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.123326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.123601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.124320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:04.125938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.136420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1212/2401] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:07:04.140353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:04.150140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364a72b-6858-4152-887e-21253aecb6af admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:04.154813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:04.155487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.155767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.156039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.157469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:04.159105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974ae11-6516-4709-ae8e-7363b396ebd5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.169239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1190/2402] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:07:04.173976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:04.184016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5cd3d-0dd1-4cb2-87a1-e0f53a5cd163 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:04.190757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:04.192238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.192536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.192816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] PATH_INFO: `/v3/projects/bb275df8683e4a1e98421dca42876377/users/2ef7a34b11a9428aa1bd93ea05f79e2f/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.213126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb275df8683e4a1e98421dca42876377, user_id=2ef7a34b11a9428aa1bd93ea05f79e2f, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:04.215838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.229697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1213/2403] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:07:04 2022] PUT /identity/v3/projects/bb275df8683e4a1e98421dca42876377/users/2ef7a34b11a9428aa1bd93ea05f79e2f/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:07:04.234788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1911ea-bb5b-4e1a-8601-bdf19b033b68 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.235156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1911ea-bb5b-4e1a-8601-bdf19b033b68 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.235437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1911ea-bb5b-4e1a-8601-bdf19b033b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.253532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ab1911ea-bb5b-4e1a-8601-bdf19b033b68 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:04.254753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ab1911ea-bb5b-4e1a-8601-bdf19b033b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:04.313740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1191/2404] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:07:04.323642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:04.325327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:04.381363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4cda3-a674-41be-9bbd-d9ec79d683eb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:04.384960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:04.385598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:04.385884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:04.386166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:04.395803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:04.397239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:04.406636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1214/2405] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:05.011493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:05.012374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:05.012655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:05.040694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] MFA Rules not processed for user `2ef7a34b11a9428aa1bd93ea05f79e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:07:05.042154 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:05.054200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:05.113550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1192/2406] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:07:05.124192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:05.125707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:05.187505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccbe21-88bc-4cfa-a402-e79b7492108d service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:05.197181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:05.197853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:05.198136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:05.198552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:05.208114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:05.209541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:05.218547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1215/2407] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:15.110051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=23 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:15.214406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:DELETE=4,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:07:56.590882 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:56.592389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:56.627088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355a83-cbdb-42ae-9a46-b841e19162a8 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:56.630697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:56.631868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:56.632153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:56.632473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:56.634247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:56.691551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:56.693252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:56.702811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1193/2408] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:07:58.586978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:07:58.605434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a3802-561a-42f7-b044-a30f24066e5c service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:07:58.609080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:07:58.609713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:07:58.609999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:07:58.610287 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:07:58.612064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:07:58.672155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:07:58.673602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:07:58.682826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1216/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:07:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:08:06.699028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=37 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:08.678585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=32 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:10.196489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:10.198044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:10.210581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a83fc-cef2-4ce9-9197-86fa3a555752 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:10.214958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:10.215684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:10.216145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:10.216603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:10.219138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:10.310070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:10.311539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca96bc3-9ac7-40be-a5e0-2b639ba01d2e service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:10.331959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1194/2410] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:08:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:08:11.513447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:11.524378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f673-6e50-4af6-91d6-6b6291c5c5e8 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:11.529423 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:11.530087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:11.530395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:11.530671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:11.532471 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:11.546545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:11.547981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:11.560237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1217/2411] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:08:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:08:12.705006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:12.705436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:12.705548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:12.761449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:08:12.762730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:12.825325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1195/2412] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 18:08:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:08:12.867073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:12.868478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:12.956432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc94c-5550-4064-af89-0550eb15aff8 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:12.961482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:12.962143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:12.962465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:12.962748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:12.971607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:12.973184 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680aeb-a061-4538-9e27-f84e18b6ba85 service swift] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:12.981754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1218/2413] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:08:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:08:13.478886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:13.489369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a6515-9b5d-4a71-a5d2-8f87fe43373a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:13.493277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:13.493922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:13.494200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:13.494505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:13.496340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:13.508257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:13.509866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0ae07-e0d1-4c40-aa65-987b1cda5f56 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:13.519458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1196/2414] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:08:19.667623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:19.668145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:19.668145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:19.669519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1219/2415] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 18:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:08:19.681118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:19.681588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:19.681689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:19.701016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:19.726940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] MFA Rules not processed for user `1f20dd9dac7a4c4bb649e484d53a4928`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:08:19.729585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:19.777069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:19.778916 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:19.827218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1197/2416] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 18:08:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:08:19.828721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a477dae-a07c-4ac4-9b28-524ce19a2d51 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:19.833573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:19.834527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:19.834815 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:19.835421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] PATH_INFO: `/v3/users/0021d731c3344934b33b5ed2b35284be` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:19.841002 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:19.843328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:19.856602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ccc17-2444-48c3-8eca-23922e30e665 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:19.860163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0021d731c3344934b33b5ed2b35284be)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:19.862538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:19.874891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:19.875630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:19.875987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:19.876341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:19.892247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:19.893710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:19.907005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1198/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:08:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:08:19.944072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0021d731c3344934b33b5ed2b35284be'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:19.952825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0021d731c3344934b33b5ed2b35284be'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:19.953520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0021d731c3344934b33b5ed2b35284be'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:19.973536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0021d731c3344934b33b5ed2b35284be'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:19.974264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0021d731c3344934b33b5ed2b35284be'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:19.995194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1220/2418] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:08:19 2022] DELETE /identity/v3/users/0021d731c3344934b33b5ed2b35284be => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:08:20.167926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:20.181823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f954b-4596-4a25-90c8-2269528f98d2 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:20.248396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:20.249251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:20.249535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:20.249811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] PATH_INFO: `/v3/projects/6a4beb48dfc14400b1872c2930ea1e11` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:20.250983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4beb48dfc14400b1872c2930ea1e11)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:20.252880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:20.297564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4beb48dfc14400b1872c2930ea1e11', 'request_id': 'req-ba488ff4-4f77-4718-a148-18642f7ce4ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.303637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4beb48dfc14400b1872c2930ea1e11', 'request_id': 'req-ba488ff4-4f77-4718-a148-18642f7ce4ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.309702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4beb48dfc14400b1872c2930ea1e11', 'request_id': 'req-ba488ff4-4f77-4718-a148-18642f7ce4ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.321940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] The token cache is being invalidate because project 6a4beb48dfc14400b1872c2930ea1e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:08:20.323272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.324243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.325818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1199/2419] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:08:20 2022] DELETE /identity/v3/projects/6a4beb48dfc14400b1872c2930ea1e11 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:08:20.330759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:20.332345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:20.351207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc122-56de-48b9-91e8-928ef4aed572 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:20.364313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:20.412850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:20.413712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:20.413991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:20.414264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] PATH_INFO: `/v3/users/d7709f1e82dd4ccb8b04e8756c31e8c0` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:20.427654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7709f1e82dd4ccb8b04e8756c31e8c0)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:20.429571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:20.474658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7709f1e82dd4ccb8b04e8756c31e8c0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.482788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7709f1e82dd4ccb8b04e8756c31e8c0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.483405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7709f1e82dd4ccb8b04e8756c31e8c0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.495928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7709f1e82dd4ccb8b04e8756c31e8c0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.496572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7709f1e82dd4ccb8b04e8756c31e8c0'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.508713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1221/2420] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:08:20 2022] DELETE /identity/v3/users/d7709f1e82dd4ccb8b04e8756c31e8c0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:08:20.681693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:20.684020 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:20.700317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba488ff4-4f77-4718-a148-18642f7ce4ec admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:20.706131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:20.761830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:20.762706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:20.762983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:20.763259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] PATH_INFO: `/v3/projects/881379e27aa0423dbe219c76acb6649a` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:20.776249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=881379e27aa0423dbe219c76acb6649a)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:20.778738 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:20.842107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881379e27aa0423dbe219c76acb6649a', 'request_id': 'req-d7d200e9-c614-4e22-9b8c-e957cc2202bb'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.849723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881379e27aa0423dbe219c76acb6649a', 'request_id': 'req-d7d200e9-c614-4e22-9b8c-e957cc2202bb'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.860660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881379e27aa0423dbe219c76acb6649a', 'request_id': 'req-d7d200e9-c614-4e22-9b8c-e957cc2202bb'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.876823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] The token cache is being invalidate because project 881379e27aa0423dbe219c76acb6649a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:08:20.877303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.878087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:08:20.879949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1200/2421] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:08:20 2022] DELETE /identity/v3/projects/881379e27aa0423dbe219c76acb6649a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:08:30.505312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:30.872844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:08:52.429431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:08:52.430983 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:52.500008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ca1fb-8837-40bb-b536-0e9dfed4750d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:08:52.546510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:08:52.547150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:08:52.547430 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:08:52.547703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:08:52.549469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:08:52.626222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:08:52.627689 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:08:52.645757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1222/2422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:08:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:09:02.643214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=81 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:09:03.593974 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:09:03.595448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:09:03.655091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d200e9-c614-4e22-9b8c-e957cc2202bb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:09:03.659854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:09:03.660876 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:09:03.661388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:09:03.661791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:09:03.664185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:09:03.674406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:09:03.675852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b939d7-7406-4202-a2b3-6a494dfbeaf1 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:09:03.683953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1201/2423] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:09:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:09:13.680303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=26 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:10:36.104605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:10:36.107623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:36.226820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c518e6-18dd-44d1-95c6-8ffe42e169a7 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:10:36.234386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:10:36.236132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:10:36.236132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:10:36.236132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:10:36.237656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:36.259684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:10:36.261780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:10:36.276262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1223/2424] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:10:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:10:40.930347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:10:40.930857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:10:40.931120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:10:40.951888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] MFA Rules not processed for user `2ed5e98a29c44296aeb921af4f521042`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:10:40.953313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:41.009341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1202/2425] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:10:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:10:41.314529 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:10:41.324131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96747-c3fb-40c5-bb4b-8fe357092246 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:10:41.327812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:10:41.328487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:10:41.328773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:10:41.329051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:10:41.330844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:41.340646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:10:41.342057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:10:41.350164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1224/2426] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:10:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:10:41.690019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:10:41.691819 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:41.767017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4915ffb-2bdf-4530-9ffd-842e2e5ebadd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:10:41.771887 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:10:41.772526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:10:41.772809 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:10:41.773086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:10:41.774946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:41.786230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:10:41.787829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:10:41.798040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1203/2427] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:10:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:10:50.283236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:10:50.297999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d85ec-9241-46eb-9539-ccc3eab38190 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:10:50.304829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:10:50.305476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:10:50.305757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:10:50.306033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:10:50.308788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:10:50.331058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:10:50.332982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:10:50.347312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1225/2428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:10:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:10:51.793935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=57 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:11:00.342750 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=46 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:11:13.927707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:11:13.929101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:11:13.942908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d7eb0-24c2-4d46-bbf1-3c79b3b0cf33 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:11:13.946488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:11:13.947148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:11:13.947431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:11:13.947706 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:11:13.949502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:11:14.010936 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:11:14.012378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:11:14.022063 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1204/2429] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:11:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:11:24.017858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=28 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:12:00.632607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:12:00.634588 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:00.645790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3bdaf-dcb4-4012-8d92-3c1158d8bd1e service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:12:00.650354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:12:00.651022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:12:00.651304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:12:00.651577 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:12:00.653357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:00.725933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:12:00.728348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:12:00.737800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1226/2430] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:12:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:12:02.021247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:12:02.033766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff58a9-4f8a-40bb-8533-c2f5e34d7ea5 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:12:02.038906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:12:02.039631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:12:02.039926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:12:02.040201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:12:02.042113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:02.054713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:12:02.056281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:12:02.066056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1205/2431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:12:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:12:05.507973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:12:05.516744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4ae0-55d7-451e-b97b-3bdfd0d9ab68 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:12:05.525859 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:12:05.526536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:12:05.526820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:12:05.527093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:12:05.528938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:05.610654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:12:05.612103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:12:05.621420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1227/2432] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:12:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:12:12.061889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:12:15.617156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=56 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:12:56.452355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:12:56.461807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d084d8-1907-40db-8917-4d6336fcc82a service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:12:56.465440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:12:56.466072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:12:56.466378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:12:56.466656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:12:56.468456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:56.528994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:12:56.530487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:12:56.540382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1206/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:12:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:12:58.163317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:12:58.172678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9a1d-4653-4701-87ce-d975c3e64024 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:12:58.176463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:12:58.177107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:12:58.177392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:12:58.177670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:12:58.179548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:12:58.191608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:12:58.193041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc522b36-32da-44ad-9bad-cd3927bea091 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:12:58.207581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1228/2434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:12:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:13:01.697179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:01.698634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:01.764189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f551982-db88-4e6f-b7cf-66c49d8f47e1 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:01.768749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:01.769611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:01.769903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:01.770156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] PATH_INFO: `/v3/users/ad1c8f646e0744a89e947061ee28255e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:01.771390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1c8f646e0744a89e947061ee28255e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:01.773939 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:01.826445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1c8f646e0744a89e947061ee28255e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:01.827131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1c8f646e0744a89e947061ee28255e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:01.849847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1c8f646e0744a89e947061ee28255e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:01.859033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1c8f646e0744a89e947061ee28255e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:01.859742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1c8f646e0744a89e947061ee28255e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:01.873535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1207/2435] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:13:01 2022] DELETE /identity/v3/users/ad1c8f646e0744a89e947061ee28255e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:13:01.878135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:01.878703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:01.878703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:01.899399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:13:01.900619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:01.963726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1229/2436] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:13:01.974635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:01.976109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:01.998091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:13:02.046837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3590e-9494-423b-807d-05725b331e85 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:02.050696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:02.051408 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:02.051690 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:02.051985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:02.062260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:02.063767 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:02.081526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1208/2437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:13:02.377136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:02.379218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:02.391979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee560-b706-482c-bc27-aba4e3a98ba6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:02.403814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:02.404646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:02.406313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:02.406313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] PATH_INFO: `/v3/projects/82c89f5009ad442985e92db48fc125df` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:02.408921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c89f5009ad442985e92db48fc125df)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:02.409298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:02.480952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c89f5009ad442985e92db48fc125df', 'request_id': 'req-02f896ef-fabc-4ce1-af2b-f0d7a9631951'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:02.486964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c89f5009ad442985e92db48fc125df', 'request_id': 'req-02f896ef-fabc-4ce1-af2b-f0d7a9631951'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:02.501491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c89f5009ad442985e92db48fc125df', 'request_id': 'req-02f896ef-fabc-4ce1-af2b-f0d7a9631951'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:02.513669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] The token cache is being invalidate because project 82c89f5009ad442985e92db48fc125df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:13:02.513953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:02.514656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:02.516205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1230/2438] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:13:02 2022] DELETE /identity/v3/projects/82c89f5009ad442985e92db48fc125df => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:13:05.517165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:05.528052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef9350-9c5e-4bef-8a50-8c1ca82c3ba4 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:05.605746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:05.606420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:05.606728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:05.607001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:05.609079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:05.684177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:05.685880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:05.696250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1209/2439] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:13:06.289958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:06.291452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:06.308963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f896ef-fabc-4ce1-af2b-f0d7a9631951 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:06.315109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:06.316849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:06.317348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:06.317846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] PATH_INFO: `/v3/users/2ef7a34b11a9428aa1bd93ea05f79e2f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:06.321277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef7a34b11a9428aa1bd93ea05f79e2f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:06.323336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:06.377643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef7a34b11a9428aa1bd93ea05f79e2f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.385770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef7a34b11a9428aa1bd93ea05f79e2f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.386435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef7a34b11a9428aa1bd93ea05f79e2f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.394634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:13:06.400435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef7a34b11a9428aa1bd93ea05f79e2f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.401749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef7a34b11a9428aa1bd93ea05f79e2f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.414816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1231/2440] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:13:06 2022] DELETE /identity/v3/users/2ef7a34b11a9428aa1bd93ea05f79e2f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:13:06.601033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:06.611380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7be4-6818-4dfb-b2bc-4279f7debf4a service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:06.683560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:06.684557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:06.684830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:06.685113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] PATH_INFO: `/v3/projects/bb275df8683e4a1e98421dca42876377` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:06.686266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb275df8683e4a1e98421dca42876377)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:06.688235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:06.701410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:13:06.739992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb275df8683e4a1e98421dca42876377', 'request_id': 'req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.749306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb275df8683e4a1e98421dca42876377', 'request_id': 'req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.753031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb275df8683e4a1e98421dca42876377', 'request_id': 'req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.768182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] The token cache is being invalidate because project bb275df8683e4a1e98421dca42876377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:13:06.768564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.769514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:13:06.771161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1210/2441] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:13:06 2022] DELETE /identity/v3/projects/bb275df8683e4a1e98421dca42876377 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:13:16.411107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=7,keystone:DELETE=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:13:16.763940 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:13:52.635442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:13:52.636924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:52.694477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f5d1-437c-4be2-81f2-138358b70e73 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:13:52.742972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:13:52.743617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:13:52.743901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:13:52.744179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:13:52.746070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:13:52.805972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:13:52.807450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:13:52.816739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1232/2442] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:14:02.812300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=68 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:14:11.593356 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:14:11.594817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:11.653822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7061e-30e3-42a1-95ab-cc7e8e0ee462 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:14:11.657543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:14:11.658184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:14:11.658505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:14:11.658780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:14:11.660626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:14:11.671786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:14:11.673216 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:14:11.682101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1211/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:14:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:14:21.678377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=26 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:16:14.418741 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:16:14.420123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:14.441427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c142ae-2259-4b85-941b-5ed4c72330f6 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:16:14.445364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:16:14.446004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:16:14.446279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:16:14.446587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:16:14.448573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:16:14.533767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:16:14.535267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:16:14.555119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1233/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:16:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:16:24.552250 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=45 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:04.880479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:04.881899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:05.029058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59094c1f-c51c-4e61-a694-339a3bfb1de1 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:05.034410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:05.035062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:05.035346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:05.035622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:05.037394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:05.049126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:05.050763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:05.060647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1212/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:17:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:17:15.056327 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=63 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:17:56.636150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:56.654935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac740f-91a3-456b-a0f7-703fb21d081b service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:56.661990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:56.662661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:56.662949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:56.663225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:56.665007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:56.749174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:56.750177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:56.768326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1234/2446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:17:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:17:58.158557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:17:58.159984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:58.183838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3a24b-1cbc-45dd-87c4-559c302f3bc9 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:17:58.188263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:17:58.188897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:17:58.189179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:17:58.189454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:17:58.191234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:17:58.270803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:17:58.272260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:17:58.290814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1213/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:17:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:06.765626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=46 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:08.288251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=46 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:10.085767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:10.105931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cd4e7-29e3-4480-863c-92745b1d2d0a service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:10.109631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:10.110311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:10.110624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:10.110899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:10.112687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:10.185609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:10.187941 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:10.199380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1235/2448] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:18:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:11.391292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:11.393713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:11.432215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57bd3b-f84e-4758-bfb1-8950e410877c service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:11.437234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:11.438441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:11.438858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:11.439688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:11.440935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:11.490223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:11.492287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:11.526611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1214/2449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:18:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:12.020558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:12.022567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:12.075344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc345c60-ecd5-4a50-a19e-beca08a9f625 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:12.082781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:12.083650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:12.085366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:12.085366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:12.087272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:12.101263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:12.103381 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:12.117437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1236/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:18:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:21.522258 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=23 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:22.112299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=56 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:52.431141 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:52.450354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb665-5d76-4472-a175-232695bf1f51 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:52.453977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:52.454660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:52.454944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:52.455218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:52.464094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:52.465526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee876e-6aea-45d4-971e-6f6483f60598 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:52.475252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1215/2451] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:18:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:58.855706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:58.857755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:58.954628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e115a-4b12-4083-8d67-725a0a21448d service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:58.959293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:58.960171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:58.960450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:58.960730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] PATH_INFO: `/v3/users/55c5ce0fa03e4c50aa5cc8b64d7ccc73` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:58.975583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c5ce0fa03e4c50aa5cc8b64d7ccc73)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:58.977541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.025795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c5ce0fa03e4c50aa5cc8b64d7ccc73'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.036695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c5ce0fa03e4c50aa5cc8b64d7ccc73'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.037350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c5ce0fa03e4c50aa5cc8b64d7ccc73'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.049806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c5ce0fa03e4c50aa5cc8b64d7ccc73'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.050515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c5ce0fa03e4c50aa5cc8b64d7ccc73'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.062827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1237/2452] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:18:58 2022] DELETE /identity/v3/users/55c5ce0fa03e4c50aa5cc8b64d7ccc73 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:18:59.067546 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.068003 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.068089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.097161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:59.098469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:59.161686 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1216/2453] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:18:59.171895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.173322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:59.218425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:18:59.235488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6f86-e640-4003-95b4-7406a8b9eed7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.240050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.240730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.241014 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.241292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.251571 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.253002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3643a-9143-407a-b3e2-d9cdfc137dd9 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.263252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1238/2454] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:18:59.554587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.556276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:59.572108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a953e-5d09-470c-b0a3-c3551b306400 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.584321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.585342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.585342 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.585630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] PATH_INFO: `/v3/projects/76e561c925824ebcb257686f304a1c1c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.600991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e561c925824ebcb257686f304a1c1c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.603125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.653999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e561c925824ebcb257686f304a1c1c', 'request_id': 'req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.662195 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e561c925824ebcb257686f304a1c1c', 'request_id': 'req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.669431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e561c925824ebcb257686f304a1c1c', 'request_id': 'req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.690854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] The token cache is being invalidate because project 76e561c925824ebcb257686f304a1c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:18:59.691139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.691757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-c84ae1bf-ea3e-4b60-8ae9-93c5df58e045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:18:59.693302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1217/2455] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:18:59 2022] DELETE /identity/v3/projects/76e561c925824ebcb257686f304a1c1c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:18:59.699022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.699346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.699596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.756759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:59.757987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:59.818971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1239/2456] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:18:59.827584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.827870 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.828142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.846585 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:18:59.847808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:18:59.856096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1218/2457] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:18:59.860420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.870874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca91c6-0061-4a26-9517-49630d0639f8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.876893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.877316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.878111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.878111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.878607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:18:59.878945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.880515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.893267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1240/2458] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:18:59.899148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.908594 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aef1e6-224f-4b40-98f5-244eb4c53289 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:18:59.913245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:18:59.913891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:18:59.914168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:18:59.914466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:18:59.915152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:18:59.917087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:18:59.957798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1219/2459] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:18:59 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:18:59.962314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:18:59.972674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748e385-7a93-4cca-b39a-9ea5f4dbbf51 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:00.030193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:00.030904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.031193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.031468 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.032165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:00.034111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:00.057946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1241/2460] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:18:59 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:19:00.059159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:00.070498 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95156277-26bd-4bd4-aea9-919995f897a4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:00.075315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:00.076000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.076278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.076550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.077270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:00.078896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:00.081235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:00.091114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1220/2461] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:19:00.094716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:00.104879 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b44f-9a0f-4f04-8d1a-27fef4a1f617 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:00.112388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:00.113472 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.113957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.114723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.115774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:00.120618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4fce4-c6af-4a1f-aa69-b1779592993a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:00.135212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1242/2462] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:19:00.139669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:00.149617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ba451-7cfa-4ed8-83a7-6c6b4a5e4a0e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:00.154295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:00.155185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.155468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.155746 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] PATH_INFO: `/v3/projects/a6b10ee6fbf445feb55336745538f31c/users/310bc28722b7465899b5138d9c9e8d54/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.170998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b10ee6fbf445feb55336745538f31c, user_id=310bc28722b7465899b5138d9c9e8d54, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:00.173658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:00.187637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1221/2463] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:19:00 2022] PUT /identity/v3/projects/a6b10ee6fbf445feb55336745538f31c/users/310bc28722b7465899b5138d9c9e8d54/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:19:00.198159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.198521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.198805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.226082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] MFA Rules not processed for user `310bc28722b7465899b5138d9c9e8d54`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:19:00.227518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:19:00.239827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:00.298867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1243/2464] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:19:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:19:00.310534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:19:00.312586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:19:00.371350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5734b-5f5d-4b91-b59f-aeffff6c7148 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:19:00.375631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:19:00.376273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:19:00.376552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:19:00.376825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:19:00.385637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:19:00.387158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:19:00.396595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1222/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:19:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:19:10.295626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=21 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:19:10.392634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:21:13.937917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:21:13.939573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:13.953980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48239-40cc-406f-a8ca-c7060872c7f3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:21:13.958279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:21:13.958993 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:21:13.959273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:21:13.959573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:21:13.961355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:21:14.024521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:21:14.026071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:21:14.036419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1244/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:21:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:21:24.031638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=28 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:08.842759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:08.844175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:08.912076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824f099-7cce-4d49-b165-48b683c4fab0 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:08.917194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:08.917844 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:08.918128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:08.918431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:08.920238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:08.932440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:08.934276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:08.944954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1223/2467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:18.940650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=32 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:40.182444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:40.187114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:40.251427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f38df-5651-4cf8-93b7-3cea1cf3cf76 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:40.255026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:40.255672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:40.255951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:40.256223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:40.258047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:40.321477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:40.322943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:40.333058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1245/2468] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:43.567993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:43.570080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:43.590981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b29c2-58ff-4b15-9839-594cb58757aa service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:43.598253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:43.598949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:43.599233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:43.599506 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:43.601420 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:43.622700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:43.624958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:43.637500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:43.638644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1224/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:43.639540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:43.653634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f028b-cd52-4a8c-9f53-208feba4df3b service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:43.659470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:43.660362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:43.660652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:43.660919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] PATH_INFO: `/v3/users/63422b2e34eb479f972a181fb37f3608` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:43.662291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63422b2e34eb479f972a181fb37f3608)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:43.664363 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:43.738174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63422b2e34eb479f972a181fb37f3608'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:43.747502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63422b2e34eb479f972a181fb37f3608'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:43.748242 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63422b2e34eb479f972a181fb37f3608'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:43.761727 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63422b2e34eb479f972a181fb37f3608'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:43.762780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a02e1534-b9d4-42eb-b925-bbb12b656139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63422b2e34eb479f972a181fb37f3608'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:43.777971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1246/2470] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:22:43 2022] DELETE /identity/v3/users/63422b2e34eb479f972a181fb37f3608 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:22:44.044330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.065200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cfcd4-1af8-40af-a568-9e4c8f18b780 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.149957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.150877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.151164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.151444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] PATH_INFO: `/v3/projects/2ed15171dff7466fab5d482c7fbb7b10` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.152697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed15171dff7466fab5d482c7fbb7b10)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.154678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.213032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed15171dff7466fab5d482c7fbb7b10', 'request_id': 'req-2994fa0d-d80b-4f34-920c-a626d83eff38'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:44.220914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed15171dff7466fab5d482c7fbb7b10', 'request_id': 'req-2994fa0d-d80b-4f34-920c-a626d83eff38'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:44.231060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed15171dff7466fab5d482c7fbb7b10', 'request_id': 'req-2994fa0d-d80b-4f34-920c-a626d83eff38'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:44.248461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] The token cache is being invalidate because project 2ed15171dff7466fab5d482c7fbb7b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:22:44.248931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:44.250317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-2994fa0d-d80b-4f34-920c-a626d83eff38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:22:44.257537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1225/2471] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:22:44 2022] DELETE /identity/v3/projects/2ed15171dff7466fab5d482c7fbb7b10 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:22:44.263687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.264079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.264444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.283854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:44.285083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:44.306564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:44.371621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1247/2472] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:22:44.374082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.375080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.375552 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.399647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:44.401158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:44.422380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1226/2473] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:44.425751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.438823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baff73d-f2f8-4aa2-b7c9-49ad5e0fbe27 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.444930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.445812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.446220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.446676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.447742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:22:44.448273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.450779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.469137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1248/2474] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:22:44.475544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.487521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f714e5-4dd7-4693-b991-bd9ac20b57c5 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.492874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.493668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.494066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.494436 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.495177 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.497135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.538234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1227/2475] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:22:44.543375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.563782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238d714-2c39-4ca9-a322-d8e5e92b6c9f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.631627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.632298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.632586 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.632865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.633565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.635603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.659166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1249/2476] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:22:44.663510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.674814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb057b5-21a1-4295-8130-d11dfb715ba0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.679608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.680280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.680563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.680838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.681564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.683531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.694516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1228/2477] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:22:44.698981 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.709311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aafa1-e8ca-48d3-8647-6f576551ffa3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.714000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.714830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.715124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.715403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.716130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.717779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29c7c2-76de-4e64-9423-0597dd6631cd admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.731396 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1250/2478] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:22:44.736279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.746226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954ad34-ce34-4700-b5f8-55a6cd8a4ff7 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.750808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.751679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.751959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.752226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] PATH_INFO: `/v3/projects/fe7ef03b108b4dc4ab72de9532ed5e46/users/dbc9cfc9f82f4a1584c8eba591021789/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.767400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7ef03b108b4dc4ab72de9532ed5e46, user_id=dbc9cfc9f82f4a1584c8eba591021789, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.770077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.785731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1229/2479] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:22:44 2022] PUT /identity/v3/projects/fe7ef03b108b4dc4ab72de9532ed5e46/users/dbc9cfc9f82f4a1584c8eba591021789/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:22:44.796040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.796334 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.796614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.823185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] MFA Rules not processed for user `dbc9cfc9f82f4a1584c8eba591021789`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:44.824422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:44.891715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1251/2480] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:44.896227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:44.910374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:44.963945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472d22e-4462-4e0c-8144-61ae73e1e5f8 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:44.969522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:44.970179 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:44.970494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:44.970774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:44.971458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:44.973385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:44.999644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1230/2481] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:22:44 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:22:45.003438 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.014141 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7435df-51c4-4c8f-9428-f80c1915f977 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.027297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:45.073010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.073680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.073962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.074237 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.075029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.076958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.100668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1252/2482] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:22:45 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:22:45.104977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.116019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f6d38-2663-4e48-897c-cd4f5e6e0cce admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.121225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.121917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.122304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.122696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.123510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.126076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.137779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1231/2483] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:22:45.143320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.159641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828caeee-7f70-4e73-b481-82890ba6ec99 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.166308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.167251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.167531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.167803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] PATH_INFO: `/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83/users/b680d4ca292d49ab915b389d2ef9ec3d/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.185855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548f9bdc736a44e8a89ac21cce8fcd83, user_id=b680d4ca292d49ab915b389d2ef9ec3d, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.189159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.204279 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1253/2484] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:22:45 2022] PUT /identity/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83/users/b680d4ca292d49ab915b389d2ef9ec3d/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:22:45.208315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.218730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce996b-1446-410d-8cbf-27f7f4cbe212 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.280438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.281118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.281396 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.281673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.282427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.284049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488e9b3-0a94-4677-b6b6-215de93a2173 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.295646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1232/2485] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:22:45.301118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.312817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235584-93f6-47f6-bfc2-fe927cecdcf8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.317779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.318737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.319025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.319312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] PATH_INFO: `/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83/users/b680d4ca292d49ab915b389d2ef9ec3d/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.321333 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=548f9bdc736a44e8a89ac21cce8fcd83, user_id=b680d4ca292d49ab915b389d2ef9ec3d, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.324057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.335625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1254/2486] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:22:45 2022] PUT /identity/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83/users/b680d4ca292d49ab915b389d2ef9ec3d/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:22:45.347074 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.347411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.347726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.374910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] MFA Rules not processed for user `b680d4ca292d49ab915b389d2ef9ec3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:22:45.376129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:45.446577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1233/2487] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:22:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:22:45.459718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.461519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:45.520303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f939d3f-f01f-4d50-a8ee-c08db267cd0b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.523984 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.524627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.524907 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.525177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.534738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.536182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.545923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1255/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:22:45.645595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:45.656240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3ff78-bbeb-4dc5-b252-34d4b1193cd8 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:45.659898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:45.660537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:45.660818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:45.661109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:45.666780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:45.721028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:45.722578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:45.731941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1234/2489] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:22:55.541464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:55.727676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=24 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:22:58.839133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:22:58.848384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf3e5c-238a-4838-bfa5-1b1c96686bc1 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:22:58.852147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:22:58.852790 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:22:58.853069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:22:58.853340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:22:58.855230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:22:58.939728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:22:58.942469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:22:58.961199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1256/2490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:22:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:23:08.952323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=28 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:23.595925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:23.597449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:23.661723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02faceb-d228-4356-a155-8315a476fc1f service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:23.665598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:23.666257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:23.666589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:23.666864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:23.668685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:23.740508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:23.741888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:23.752673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1235/2491] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:23:27.485241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:27.487606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:27.516176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d4744-0f6d-4e9d-9984-943333876c96 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:27.522404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:27.523445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:27.523861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:27.524191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] PATH_INFO: `/v3/users/de0ef30352514ef9bb4559473e096ea6` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:27.526113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0ef30352514ef9bb4559473e096ea6)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:27.530020 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:27.610884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0ef30352514ef9bb4559473e096ea6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:27.621232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0ef30352514ef9bb4559473e096ea6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:27.622043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0ef30352514ef9bb4559473e096ea6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:27.638244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0ef30352514ef9bb4559473e096ea6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:27.639142 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cf5f69dc-d30a-4a43-aab7-610ce1e8dd7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0ef30352514ef9bb4559473e096ea6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:27.653873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1257/2492] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:23:27 2022] DELETE /identity/v3/users/de0ef30352514ef9bb4559473e096ea6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:27.868449 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:27.883071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489183c8-bc48-4486-9c95-60e94494911a service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:27.973610 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:27.974435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:27.974724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:27.975000 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] PATH_INFO: `/v3/projects/221481e35182443995ea17136383f6f4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:27.976271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=221481e35182443995ea17136383f6f4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:27.978226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.032753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221481e35182443995ea17136383f6f4', 'request_id': 'req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:28.035562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:28.045493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221481e35182443995ea17136383f6f4', 'request_id': 'req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:28.053102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221481e35182443995ea17136383f6f4', 'request_id': 'req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:28.068108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] The token cache is being invalidate because project 221481e35182443995ea17136383f6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:23:28.068536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:28.069315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-8c85e212-5a0e-4bc7-9c2f-ad758b0fa61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:28.071084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1236/2493] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:23:27 2022] DELETE /identity/v3/projects/221481e35182443995ea17136383f6f4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:23:28.077953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.078505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.078882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.098888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:28.100149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:28.165127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1258/2494] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:28.174317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.174968 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.175465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.200202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:28.201511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:28.210071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1237/2495] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:28.214754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.225625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedffef-6c82-4517-8189-7da24da72131 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.230568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.231239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.231658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.232060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.232802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:23:28.233110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.234770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.254030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1259/2496] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:23:28.260958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.277961 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff2cc5-a0af-48bb-bed1-117662f2c02c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.286570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.287717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.288215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.288708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.289992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.293703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.338364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1238/2497] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:28.344489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.355661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03a9ad-b119-4235-a729-86abbaeba638 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.426404 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.427692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.428062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.428564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.429883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.432597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.440623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:28.461425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1260/2498] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:28.466111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.480739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c03a0-5d07-4a81-a62f-3195963461fb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.488073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.490254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.490640 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.490984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.491810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.493701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.510812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1239/2499] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:23:28.512285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.526715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9b3d9-32e5-4ea7-8e85-bfe82af85497 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.532632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.533318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.533602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.533912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.534648 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.536285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f9e6c-351b-4539-8668-5c5e5e665910 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.549258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1261/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:23:28.553670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.565737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa2968-d380-48b2-9cdf-cf983ca96e11 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.570967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.571898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.572181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.572459 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] PATH_INFO: `/v3/projects/599988efb07f47739b243711f84383f1/users/5c09b6d9d7984347a91ae89fc8437188/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.589445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599988efb07f47739b243711f84383f1, user_id=5c09b6d9d7984347a91ae89fc8437188, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.592297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.609805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1240/2501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:28 2022] PUT /identity/v3/projects/599988efb07f47739b243711f84383f1/users/5c09b6d9d7984347a91ae89fc8437188/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:28.623056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.623229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.624035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.669776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] MFA Rules not processed for user `5c09b6d9d7984347a91ae89fc8437188`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:28.671087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:28.743098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1262/2502] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:28.748741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.856785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba9720-85c3-4ce4-b1ce-d10c2da662e1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.864439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.865547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.866044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.866572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.867836 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:28.871326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:28.910759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1241/2503] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:28.915078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:28.926775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1dbab-ad6b-4e35-82fa-93002b3e01ec None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:28.998490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:28.998490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:28.998490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:28.998954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:28.999591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.001640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.028405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1263/2504] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:23:28 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:23:29.033442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.045586 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01869309-8e51-4a1a-b3eb-453f052bafd2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.050634 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.051344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.051637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.051925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.052668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.054300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.067081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1242/2505] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:23:29.073169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.086053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92333466-6d09-4637-b77a-75e6f5768643 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.092405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.093528 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.093905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.094275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] PATH_INFO: `/v3/projects/560e38ad3ec048fcabda11633285cf70/users/30ecd63d88854b6b81ac533639cc00cd/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.111944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=560e38ad3ec048fcabda11633285cf70, user_id=30ecd63d88854b6b81ac533639cc00cd, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.115105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.130554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1264/2506] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:29 2022] PUT /identity/v3/projects/560e38ad3ec048fcabda11633285cf70/users/30ecd63d88854b6b81ac533639cc00cd/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:29.135004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.145757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5286691-2427-4b1a-bb90-4f41c818c357 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.198689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:29.212856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.213540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.213827 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.214107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.214907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.216584 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0db554-8876-4d3d-9b74-27fdee72562f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.227548 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1243/2507] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:23:29.233389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.245253 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ae5e-8c0c-4c74-85c7-76469d95ee05 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.251058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.251884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.252163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.252434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] PATH_INFO: `/v3/projects/560e38ad3ec048fcabda11633285cf70/users/30ecd63d88854b6b81ac533639cc00cd/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.254619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] RBAC: Authorizing `identity:create_grant(project_id=560e38ad3ec048fcabda11633285cf70, user_id=30ecd63d88854b6b81ac533639cc00cd, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.257362 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.271232 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1265/2508] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:29 2022] PUT /identity/v3/projects/560e38ad3ec048fcabda11633285cf70/users/30ecd63d88854b6b81ac533639cc00cd/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:29.283829 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.284207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.284533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.322653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] MFA Rules not processed for user `30ecd63d88854b6b81ac533639cc00cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:29.324156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:29.416581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1244/2509] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:29 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:29.434191 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.436098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:29.459953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:29.501598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96bd6e-a327-4f06-bf03-9f0d6218624a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.505642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.506398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.506729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.507115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.517255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.518917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.530130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1266/2510] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:29 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:29.632476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:29.643333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a6d51-c6ed-4c02-b4ad-fb08c672e2dc None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:29.647517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:29.648223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:29.648466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:29.648736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:29.748845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:29.751474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bca3d-042f-4a0a-9c21-517fd69aef49 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:29.778243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1245/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:23:32.665019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:32.736955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1785c13-308a-40b5-a695-e57883e12330 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:32.744199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:32.745736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:32.746233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:32.746742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] PATH_INFO: `/v3/users/5c09b6d9d7984347a91ae89fc8437188` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:32.748874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c09b6d9d7984347a91ae89fc8437188)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:32.752664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:32.810240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c09b6d9d7984347a91ae89fc8437188'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:32.819039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c09b6d9d7984347a91ae89fc8437188'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:32.819675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c09b6d9d7984347a91ae89fc8437188'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:32.839162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c09b6d9d7984347a91ae89fc8437188'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:32.840303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c09b6d9d7984347a91ae89fc8437188'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:32.862694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1267/2512] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:23:32 2022] DELETE /identity/v3/users/5c09b6d9d7984347a91ae89fc8437188 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:32.869253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:32.870056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:32.870263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:32.893386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:32.895410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:32.984264 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1246/2513] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:23:32.995336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:32.996875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:33.049393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:33.066578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462ff65-5ab1-43f8-b9c3-9028664f27cc admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:33.071029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:33.071840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:33.072207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:33.072565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:33.083493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:33.084971 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:33.095620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1268/2514] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:33.389681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:33.392269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:33.411204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd85c2-8f60-48ca-b9eb-c5cc34aa0e25 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:33.418281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:33.420450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:33.420942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:33.421423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] PATH_INFO: `/v3/projects/599988efb07f47739b243711f84383f1` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:33.423442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] RBAC: Authorizing `identity:delete_project(project_id=599988efb07f47739b243711f84383f1)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:33.425630 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:33.433089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:33.478204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599988efb07f47739b243711f84383f1', 'request_id': 'req-1ed5fa71-51c8-48e0-b589-b0569135397a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.486112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599988efb07f47739b243711f84383f1', 'request_id': 'req-1ed5fa71-51c8-48e0-b589-b0569135397a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.492798 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599988efb07f47739b243711f84383f1', 'request_id': 'req-1ed5fa71-51c8-48e0-b589-b0569135397a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.506238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] The token cache is being invalidate because project 599988efb07f47739b243711f84383f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:23:33.506625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.507381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.509102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1247/2515] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:23:33 2022] DELETE /identity/v3/projects/599988efb07f47739b243711f84383f1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:33.513858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:33.523757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebef71-93b0-4cfb-911a-b49aa19197b6 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:33.589760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:33.591302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:33.591578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:33.591858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] PATH_INFO: `/v3/users/30ecd63d88854b6b81ac533639cc00cd` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:33.593317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ecd63d88854b6b81ac533639cc00cd)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:33.595368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:33.652980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ecd63d88854b6b81ac533639cc00cd'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.662196 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ecd63d88854b6b81ac533639cc00cd'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.663030 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ecd63d88854b6b81ac533639cc00cd'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.677999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ecd63d88854b6b81ac533639cc00cd'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.679079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-db756cca-6db5-4844-912c-1bc461b57859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ecd63d88854b6b81ac533639cc00cd'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:33.693856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1269/2516] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:23:33 2022] DELETE /identity/v3/users/30ecd63d88854b6b81ac533639cc00cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:34.044641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.047045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:34.069576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5fa71-51c8-48e0-b589-b0569135397a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.167193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.168591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.169859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.170376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] PATH_INFO: `/v3/projects/560e38ad3ec048fcabda11633285cf70` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.172306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=560e38ad3ec048fcabda11633285cf70)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.174409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.229990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560e38ad3ec048fcabda11633285cf70', 'request_id': 'req-018489ed-672e-44c2-a29d-062886484909'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:34.242149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560e38ad3ec048fcabda11633285cf70', 'request_id': 'req-018489ed-672e-44c2-a29d-062886484909'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:34.251894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560e38ad3ec048fcabda11633285cf70', 'request_id': 'req-018489ed-672e-44c2-a29d-062886484909'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:34.269274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] The token cache is being invalidate because project 560e38ad3ec048fcabda11633285cf70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:23:34.269561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:34.270221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-018489ed-672e-44c2-a29d-062886484909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:23:34.271874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1248/2517] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:23:34 2022] DELETE /identity/v3/projects/560e38ad3ec048fcabda11633285cf70 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:34.276890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.277179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.277461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.297513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:34.298866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:34.369252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1270/2518] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:34.377366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.377668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.377953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.397998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:34.399680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:34.408719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1249/2519] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:34.413942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.420351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:34.425716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2873f-77f0-441c-844e-eba5ae5b6716 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.433129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.433830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.434136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.434440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.435171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:23:34.435747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.437676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.451090 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1271/2520] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:23:34.455837 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.470348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c9c2-f794-434e-95cb-d5545f67e80a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.475486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.476166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.476474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.476749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.477446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.479427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.495323 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:34.510108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1250/2521] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:34.514980 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.525545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c2341-6863-439f-913f-0f7fd5487d4e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.587994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.588676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.588962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.589240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.589947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.592772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.614087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1272/2522] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:34.617992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.629658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c36185-2196-47a0-b0f5-4cb1c011e703 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.635635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.636314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.636598 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.636869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.638162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.639830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.652841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1251/2523] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:23:34.656985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.671219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65179d20-d9ad-45e8-b138-f6e8702b81eb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.676059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.676752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.677038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.677317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.678072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.679721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6db36-118f-404d-8d65-998995ec3424 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.691267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1273/2524] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:23:34.695867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.706839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f53b-185c-423e-9578-a03c5f139e8c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.711707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.712591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.712879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.713156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] PATH_INFO: `/v3/projects/9a48799b3c514bb7a154029b8cf29ab7/users/419a0c0359014f0e9f2226516e60ee50/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.731308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a48799b3c514bb7a154029b8cf29ab7, user_id=419a0c0359014f0e9f2226516e60ee50, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.734030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:34.748950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1252/2525] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:34 2022] PUT /identity/v3/projects/9a48799b3c514bb7a154029b8cf29ab7/users/419a0c0359014f0e9f2226516e60ee50/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:23:34.759326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.759584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.759866 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.789116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] MFA Rules not processed for user `419a0c0359014f0e9f2226516e60ee50`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:34.790379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:34.879898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1274/2526] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:34.885047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:34.969001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67510630-b9bb-4a70-be26-f22c3289eb1c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:34.974105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:34.974821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:34.975100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:34.975394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:34.976081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:34.977997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.006380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1253/2527] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:23:34 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:23:35.011260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.022857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab20ed-075c-4171-bbbe-147f57f6341a None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.087712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.088384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.088671 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.088943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.089641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.091632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.099168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:35.118902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1275/2528] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:23:35 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:23:35.127434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.138568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036aecc-907b-463d-8ef8-cb6720641b03 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.143384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.144132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.148503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.148807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.149549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.151281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.163821 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1254/2529] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:23:35.169235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.186249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23c310-c765-4fd9-a20d-6b76962818ef admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.191572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.192389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.192680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.192942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] PATH_INFO: `/v3/projects/1f72d1860d4e44c7b85754fafe22ba46/users/014999ad010a4df081f8ea5b4196caf6/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.209825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f72d1860d4e44c7b85754fafe22ba46, user_id=014999ad010a4df081f8ea5b4196caf6, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.212557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.226976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1276/2530] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:35 2022] PUT /identity/v3/projects/1f72d1860d4e44c7b85754fafe22ba46/users/014999ad010a4df081f8ea5b4196caf6/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:35.231243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.242156 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9a082-baf1-4e40-88d7-a022a2419ce6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.302223 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.302924 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.303205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.303478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.304196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.305813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab629a-c4b2-4243-8b77-7550a495527d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.323847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1255/2531] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:23:35.328846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.339476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b40158-49ce-4e3e-8835-503837419453 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.344165 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.345028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.345310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.345582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] PATH_INFO: `/v3/projects/1f72d1860d4e44c7b85754fafe22ba46/users/014999ad010a4df081f8ea5b4196caf6/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.347789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f72d1860d4e44c7b85754fafe22ba46, user_id=014999ad010a4df081f8ea5b4196caf6, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.350474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.362565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1277/2532] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:23:35 2022] PUT /identity/v3/projects/1f72d1860d4e44c7b85754fafe22ba46/users/014999ad010a4df081f8ea5b4196caf6/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:23:35.374667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.374957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.375236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.403240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] MFA Rules not processed for user `014999ad010a4df081f8ea5b4196caf6`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:23:35.404492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:35.425975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:35.478240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1256/2533] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:23:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:23:35.489668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:23:35.492547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:23:35.582010 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919e7d2-8567-4499-bf35-c7436cd18d4c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:23:35.586506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:23:35.587192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:23:35.587483 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:23:35.587810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:23:35.665228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:23:35.666809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:23:35.677878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1278/2534] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:23:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:23:45.475167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=20 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:23:45.673068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=5,keystone:SELECT=65 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:24:01.322040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:24:01.333059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a0015-7edf-411f-9c6f-7a70e31164cf None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:24:01.337735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:24:01.338604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:24:01.339054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:24:01.340244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:24:01.342838 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:24:01.425615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:24:01.427135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9e51f-8ac8-4c56-ac6d-a75660091526 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:24:01.437744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1257/2535] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:24:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:24:11.433360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=28 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:00.562672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:00.564260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:00.630665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66b5cf-a5e1-479d-b340-7ed17cb10fcf service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:00.635794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:00.636657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:00.636943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:00.637219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] PATH_INFO: `/v3/users/310bc28722b7465899b5138d9c9e8d54` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:00.638426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=310bc28722b7465899b5138d9c9e8d54)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:00.640380 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:00.686868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '310bc28722b7465899b5138d9c9e8d54'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:00.693826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '310bc28722b7465899b5138d9c9e8d54'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:00.694484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '310bc28722b7465899b5138d9c9e8d54'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:00.706716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '310bc28722b7465899b5138d9c9e8d54'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:00.707348 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '310bc28722b7465899b5138d9c9e8d54'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:00.719919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1279/2536] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:25:00 2022] DELETE /identity/v3/users/310bc28722b7465899b5138d9c9e8d54 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:25:00.724376 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:00.724874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:00.724962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:00.743533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:00.744718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:00.802231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1258/2537] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:00.812707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:00.814124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:00.871075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d832d31-9e40-419f-a63d-07c9097f9a22 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:00.874761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:00.875406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:00.875698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:00.875967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:00.884998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:00.886466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439e5d-c90c-45bc-93b9-8e01215902a7 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:00.895999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1280/2538] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:25:01.170875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.172362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:01.185681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c075c-3e80-4254-a59a-99fdd2e4d16c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.192144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.193213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.193384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.193656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] PATH_INFO: `/v3/projects/a6b10ee6fbf445feb55336745538f31c` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.195875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b10ee6fbf445feb55336745538f31c)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.198168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.256013 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b10ee6fbf445feb55336745538f31c', 'request_id': 'req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:01.262305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b10ee6fbf445feb55336745538f31c', 'request_id': 'req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:01.270674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b10ee6fbf445feb55336745538f31c', 'request_id': 'req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:01.283602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] The token cache is being invalidate because project a6b10ee6fbf445feb55336745538f31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:25:01.283889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:01.284503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d8d80b55-a4c8-4d3d-9125-5abb5312ee94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:25:01.286254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1259/2539] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:25:01 2022] DELETE /identity/v3/projects/a6b10ee6fbf445feb55336745538f31c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:25:01.293035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.293331 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.293616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.311652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:01.312881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:01.333672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:01.378212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1281/2540] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:01.386159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.386482 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.386763 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.405637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:01.406899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:01.415902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1260/2541] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:01.420290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.430372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eb39d-c0ca-48f7-bb64-b3495e36186e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.434909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.435562 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.435839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.436112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.436830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:25:01.437133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.438765 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.450744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1282/2542] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:25:01.455263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.465059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9453d53-4896-4bd8-b9de-ee6642d03ae4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.469741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.470423 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.470711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.470986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.471668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.473572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.499103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1261/2543] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:25:01 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:25:01.503163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.513187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa327-eb88-49cc-8b4d-e8e5110e119d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.573202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.573905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.574135 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.574436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.575126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.577051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.601091 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1283/2544] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:25:01 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:25:01.605550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.616393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6d90-61b6-48da-9bcb-270ffb12c47c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.621080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.621776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.622056 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.622344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.623072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.624660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.638294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1262/2545] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:25:01.639853 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.651094 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81b2b2-890e-475f-aced-b6a6ba11e261 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.656112 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.656780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.657066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.657349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.658074 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.659735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67acd64-052d-49a2-b715-3ba2c8a66246 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.671434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1284/2546] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:25:01.675673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.686060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f40328-9fa8-478b-8564-9f5ab44ee139 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.690704 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.691581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.691865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.692144 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] PATH_INFO: `/v3/projects/a262c7be2bc040eea71787e95b11c92f/users/82765721711e484d8bc87340177657cc/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.707753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a262c7be2bc040eea71787e95b11c92f, user_id=82765721711e484d8bc87340177657cc, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.710456 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.723977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1263/2547] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:25:01 2022] PUT /identity/v3/projects/a262c7be2bc040eea71787e95b11c92f/users/82765721711e484d8bc87340177657cc/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:25:01.729298 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004a1b-9162-41e4-8a73-c8da8f051c79 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.729622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004a1b-9162-41e4-8a73-c8da8f051c79 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.729904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004a1b-9162-41e4-8a73-c8da8f051c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.749437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8c004a1b-9162-41e4-8a73-c8da8f051c79 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:01.750619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8c004a1b-9162-41e4-8a73-c8da8f051c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:01.810878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1285/2548] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:25:01.820991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:01.822751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:01.845583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:01.884675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8e621-97f4-461b-bfe1-41a92b69c2d1 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:01.888303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:01.888942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:01.889221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:01.889493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:01.898437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:01.900477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:01.910028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1264/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:25:06.255998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.256496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.256592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.288996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] MFA Rules not processed for user `82765721711e484d8bc87340177657cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:06.290233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:06.308554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:06.363009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1286/2550] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:06.367501 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.378657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab77e9-2536-45f1-b332-0c1dd5c0e381 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.386091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.388742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.388742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.388742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.388742 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.390049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.419437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1265/2551] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:25:06 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:25:06.423710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.436536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a643f51-90d1-4abf-b4b0-b262440c9943 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.510850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.511543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.511824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.512123 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.512839 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.514883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.538395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1287/2552] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:25:06 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:25:06.540810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.553989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d3b76-f53e-48c4-ac3d-fff2b48c70c9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.559281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.559950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.560228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.560502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.561218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.562853 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.575669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1266/2553] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:25:06.579520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.590522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06040738-1a64-44cf-82f0-c21778aec3b4 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.595617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.596517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.596804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.597082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] PATH_INFO: `/v3/projects/7144a4c42655475e890db2a23d0517ac/users/e85c4d8a2c8441f69886848937b770f6/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.622748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7144a4c42655475e890db2a23d0517ac, user_id=e85c4d8a2c8441f69886848937b770f6, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.625548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.640134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1288/2554] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:25:06 2022] PUT /identity/v3/projects/7144a4c42655475e890db2a23d0517ac/users/e85c4d8a2c8441f69886848937b770f6/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:25:06.644045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.654184 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3f4b-b2d7-4bf2-ad5b-149e0b70bf83 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.716885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.717564 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.717852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.718139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.719031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.720673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a602e0-bc4a-408b-b29b-e1f6284390a6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.731785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1267/2555] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:25:06.735985 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:06.747343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f65f9-07ea-43d3-b911-d109646224f1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:06.752127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:06.753035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:06.753317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:06.753595 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] PATH_INFO: `/v3/projects/7144a4c42655475e890db2a23d0517ac/users/e85c4d8a2c8441f69886848937b770f6/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:06.755678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7144a4c42655475e890db2a23d0517ac, user_id=e85c4d8a2c8441f69886848937b770f6, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:06.759007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:06.770802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1289/2556] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:25:06 2022] PUT /identity/v3/projects/7144a4c42655475e890db2a23d0517ac/users/e85c4d8a2c8441f69886848937b770f6/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:25:11.562281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9c4f9-a044-4daf-b3db-c8f8c3281d85 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:11.562831 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9c4f9-a044-4daf-b3db-c8f8c3281d85 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:11.563095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9c4f9-a044-4daf-b3db-c8f8c3281d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:11.591699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-15e9c4f9-a044-4daf-b3db-c8f8c3281d85 None None] MFA Rules not processed for user `e85c4d8a2c8441f69886848937b770f6`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:11.592996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-15e9c4f9-a044-4daf-b3db-c8f8c3281d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:11.674035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1268/2557] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:11.687829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:25:11.690009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:11.771269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01257130-496a-493c-ab90-ba48f911fae5 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:25:11.777673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:25:11.778341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:11.778636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:11.778909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:11.785225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:11.843186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:25:11.845018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:25:11.854994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1290/2558] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:25:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:25:12.926507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:25:12.926960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:25:12.927200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:25:12.951479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] MFA Rules not processed for user `82765721711e484d8bc87340177657cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:25:12.953196 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:25:12.964183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1269/2559] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:25:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:25:21.851110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=24 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:25:22.943934 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:09.688153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:09.689706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:09.774239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad65e51-5efc-4515-aa5b-d5bf4b194261 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:09.779409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:09.781959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:09.782276 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:09.783450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:09.783737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:09.784012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] PATH_INFO: `/v3/users/dbc9cfc9f82f4a1584c8eba591021789` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:09.785226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc9cfc9f82f4a1584c8eba591021789)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:09.787844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:09.843359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc9cfc9f82f4a1584c8eba591021789'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:09.844637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c030322-8a7a-4b29-a874-6173b8a25c75 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:09.850639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:09.851313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:09.851477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc9cfc9f82f4a1584c8eba591021789'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:09.851674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:09.852156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc9cfc9f82f4a1584c8eba591021789'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:09.852283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:09.854134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:09.865278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc9cfc9f82f4a1584c8eba591021789'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:09.865637 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-0c5484cb-d8dd-49bf-a568-dc6ca6b2bdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc9cfc9f82f4a1584c8eba591021789'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:09.878943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1291/2560] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:09 2022] DELETE /identity/v3/users/dbc9cfc9f82f4a1584c8eba591021789 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:09.883593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:09.883940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:09.884262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:09.902432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:09.903647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:09.918230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:09.919624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:09.929514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1270/2561] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:26:09.970060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1292/2562] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:09.981688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:09.983189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:10.054444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16dbe-b493-4952-a80c-543ff7c7db34 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:10.108100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:10.108783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:10.109068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:10.109372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:10.119194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:10.120714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:10.138982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1271/2563] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:10.456003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:10.457465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:10.469885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd7e1-345d-4d5d-a158-083d222c0fd9 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:10.474751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:10.475633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:10.475915 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:10.476185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] PATH_INFO: `/v3/projects/fe7ef03b108b4dc4ab72de9532ed5e46` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:10.477421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe7ef03b108b4dc4ab72de9532ed5e46)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:10.484647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:10.506273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:10.542470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe7ef03b108b4dc4ab72de9532ed5e46', 'request_id': 'req-37951048-2671-477f-af8c-f3e97214395e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.548160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe7ef03b108b4dc4ab72de9532ed5e46', 'request_id': 'req-37951048-2671-477f-af8c-f3e97214395e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.557766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe7ef03b108b4dc4ab72de9532ed5e46', 'request_id': 'req-37951048-2671-477f-af8c-f3e97214395e'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.571889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] The token cache is being invalidate because project fe7ef03b108b4dc4ab72de9532ed5e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:10.572320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.572900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.574696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1293/2564] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:10 2022] DELETE /identity/v3/projects/fe7ef03b108b4dc4ab72de9532ed5e46 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:10.578908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:10.589229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1bd59-1337-43e7-a5c1-e880196f7ca4 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:10.594512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:10.648178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:10.649065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:10.649368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:10.649638 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] PATH_INFO: `/v3/users/b680d4ca292d49ab915b389d2ef9ec3d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:10.650849 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b680d4ca292d49ab915b389d2ef9ec3d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:10.652764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:10.697999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b680d4ca292d49ab915b389d2ef9ec3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.698724 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b680d4ca292d49ab915b389d2ef9ec3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.710856 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b680d4ca292d49ab915b389d2ef9ec3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.719204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b680d4ca292d49ab915b389d2ef9ec3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.719824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-44c519c7-ee47-4149-bcb2-7f10ea5a0f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b680d4ca292d49ab915b389d2ef9ec3d'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:10.733337 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1272/2565] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:10 2022] DELETE /identity/v3/users/b680d4ca292d49ab915b389d2ef9ec3d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:11.003702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.005164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:11.020185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951048-2671-477f-af8c-f3e97214395e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.080809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.081678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.081961 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.082235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] PATH_INFO: `/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.084138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=548f9bdc736a44e8a89ac21cce8fcd83)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.086054 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.132715 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548f9bdc736a44e8a89ac21cce8fcd83', 'request_id': 'req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:11.138108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548f9bdc736a44e8a89ac21cce8fcd83', 'request_id': 'req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:11.147446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548f9bdc736a44e8a89ac21cce8fcd83', 'request_id': 'req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:11.158487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] The token cache is being invalidate because project 548f9bdc736a44e8a89ac21cce8fcd83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:11.158774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:11.159405 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-3a8a9b0b-3444-46b9-adfb-1bffd9e351dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:11.161789 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1294/2566] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:11 2022] DELETE /identity/v3/projects/548f9bdc736a44e8a89ac21cce8fcd83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:11.166718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.167199 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.167287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.185551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:11.186910 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:11.249713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1273/2567] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:11.305487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.305807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.306088 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.325171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:11.326417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:11.335179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1295/2568] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:11.339353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.349886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb357f51-f9bb-4017-847b-e7a1115b46d6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.354359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.355026 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.355292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.355563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.356274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:26:11.356575 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.358167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.369756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1274/2569] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:26:11.374118 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.383406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dde2d-7f4e-4d3f-b3d2-95ead58d9f9e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.388098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.388744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.389035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.389312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.390001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.391942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.418831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1296/2570] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:11.422830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.433219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b110a5-4f54-42ee-9b7f-107807394ece admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.444807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:11.492151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.492818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.493100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.493401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.494108 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.496084 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.520550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1275/2571] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:26:11.521734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.528444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:11.532300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e56cd1-9699-4ae0-94a8-33738d45f20e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.538635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.539530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.540011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.540301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.541236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.542885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.553895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1297/2572] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:26:11.557946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.569077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71633aca-5915-4608-a3aa-dc515a3bebaf admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.573639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.574277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.574702 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.574985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.575698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.577307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2cb834-f33c-4ce7-b609-491e237cb3e0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.588238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1276/2573] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:11.592051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.601928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c61938-6296-4959-b52a-eb479abe65ec admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.606519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.607385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.607665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.607934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] PATH_INFO: `/v3/projects/b506dfbb49024e149b8431dd73594e53/users/cc7a0498b4464440b7ea5a1a88a0a7e1/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.622122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b506dfbb49024e149b8431dd73594e53, user_id=cc7a0498b4464440b7ea5a1a88a0a7e1, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.624811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.638480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1298/2574] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:11 2022] PUT /identity/v3/projects/b506dfbb49024e149b8431dd73594e53/users/cc7a0498b4464440b7ea5a1a88a0a7e1/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:11.649221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.649508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.649783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.675618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] MFA Rules not processed for user `cc7a0498b4464440b7ea5a1a88a0a7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:11.676854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:11.743269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1277/2575] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:11.747881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.810140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296c48b-8151-4221-b036-8ff662694733 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.814798 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.815440 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.815723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.815996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.816679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.819106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.843897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1299/2576] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:11.847885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.858603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91c4b-789f-4a97-aac3-0facbb5ca875 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.915471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.916134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.916414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.916687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.917393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.919340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.941505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1278/2577] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:11 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:11.946203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.957338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4831200-1ec2-4c5c-86ea-23ce857a3bfd admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.962536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.963714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.964016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.964294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:11.965029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:11.966693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:11.977828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1300/2578] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:11.982046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:11.993080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4c322-ce08-4bdf-b14b-07acbe1085cb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:11.997689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:11.998590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:11.998871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:11.999159 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] PATH_INFO: `/v3/projects/d725470b8fbf40d6b50dc10dc7d47878/users/ef1e0d0f1c2e4414b8b71887689e7e51/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.011033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:12.016234 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d725470b8fbf40d6b50dc10dc7d47878, user_id=ef1e0d0f1c2e4414b8b71887689e7e51, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:12.019161 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:12.033994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1279/2579] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:11 2022] PUT /identity/v3/projects/d725470b8fbf40d6b50dc10dc7d47878/users/ef1e0d0f1c2e4414b8b71887689e7e51/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:12.038004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:12.048039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6952fa5-8a0b-4749-b9fe-7eaf6f0b7951 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:12.104635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:12.105220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.105502 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.105773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.106522 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:12.108099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664388ad-a8e4-4b92-aa2b-5a59c715fd22 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:12.121806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1301/2580] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:12.123240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:12.134171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b3e9c-74c6-4655-86f2-1273ad89e111 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:12.139109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:12.139989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.140275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.140547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] PATH_INFO: `/v3/projects/d725470b8fbf40d6b50dc10dc7d47878/users/ef1e0d0f1c2e4414b8b71887689e7e51/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.142649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d725470b8fbf40d6b50dc10dc7d47878, user_id=ef1e0d0f1c2e4414b8b71887689e7e51, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:12.145305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:12.156042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1280/2581] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:12 2022] PUT /identity/v3/projects/d725470b8fbf40d6b50dc10dc7d47878/users/ef1e0d0f1c2e4414b8b71887689e7e51/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:12.167016 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.167308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.167587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.194646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] MFA Rules not processed for user `ef1e0d0f1c2e4414b8b71887689e7e51`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:12.195864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:12.224338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:12.272871 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1302/2582] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:12.283464 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:12.285228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:12.346987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b129-17f2-4e66-9d31-422f9a5eddfb admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:12.352189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:12.353093 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.353494 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.354009 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.365775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:12.367115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002027-82d7-4aa9-b1b6-325957918438 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:12.385230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1281/2583] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:12.744152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:12.805491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ad31-db94-4d5a-91e2-54f3e6829e72 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:12.810002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:12.810897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.811179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.811452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] PATH_INFO: `/v3/users/cc7a0498b4464440b7ea5a1a88a0a7e1` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.812628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7a0498b4464440b7ea5a1a88a0a7e1)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:12.814570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:12.859386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7a0498b4464440b7ea5a1a88a0a7e1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:12.867132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7a0498b4464440b7ea5a1a88a0a7e1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:12.867751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7a0498b4464440b7ea5a1a88a0a7e1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:12.879896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7a0498b4464440b7ea5a1a88a0a7e1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:12.880593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7a0498b4464440b7ea5a1a88a0a7e1'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:12.892805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1303/2584] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:12 2022] DELETE /identity/v3/users/cc7a0498b4464440b7ea5a1a88a0a7e1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:12.897834 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:12.898123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:12.898427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:12.916019 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:12.917252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:12.975372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1282/2585] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:12.984832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:12.986278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:13.034266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:13.046676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88e33c-ab16-4cb6-a0ab-8e051671f604 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:13.050598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:13.051268 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:13.051547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:13.051814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:13.061355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:13.062819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:13.072710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1304/2586] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:13.342152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:13.343633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:13.359151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fde79b-3f30-4265-b375-10de6e81dda3 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:13.364295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:13.365175 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:13.365609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:13.365736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] PATH_INFO: `/v3/projects/b506dfbb49024e149b8431dd73594e53` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:13.367004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b506dfbb49024e149b8431dd73594e53)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:13.368943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:13.417766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b506dfbb49024e149b8431dd73594e53', 'request_id': 'req-87acaff0-0a0c-4410-afb3-66e954a572b6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.423907 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b506dfbb49024e149b8431dd73594e53', 'request_id': 'req-87acaff0-0a0c-4410-afb3-66e954a572b6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.429671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b506dfbb49024e149b8431dd73594e53', 'request_id': 'req-87acaff0-0a0c-4410-afb3-66e954a572b6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.441897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] The token cache is being invalidate because project b506dfbb49024e149b8431dd73594e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:13.442192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.442898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.444458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1283/2587] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:13 2022] DELETE /identity/v3/projects/b506dfbb49024e149b8431dd73594e53 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:13.448452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:13.458647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec1df7-73b6-40d0-9095-776043048cb7 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:13.514465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:13.515326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:13.515610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:13.515887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] PATH_INFO: `/v3/users/ef1e0d0f1c2e4414b8b71887689e7e51` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:13.517066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1e0d0f1c2e4414b8b71887689e7e51)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:13.519022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:13.564477 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1e0d0f1c2e4414b8b71887689e7e51'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.571653 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1e0d0f1c2e4414b8b71887689e7e51'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.572269 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1e0d0f1c2e4414b8b71887689e7e51'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.584431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1e0d0f1c2e4414b8b71887689e7e51'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.585058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-15daddc5-acd4-42e3-a2b8-2da1e0a593d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1e0d0f1c2e4414b8b71887689e7e51'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:13.596964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1305/2588] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:13 2022] DELETE /identity/v3/users/ef1e0d0f1c2e4414b8b71887689e7e51 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:13.870325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:13.871803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:13.881570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:13.892534 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87acaff0-0a0c-4410-afb3-66e954a572b6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:13.949732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:13.950557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:13.950839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:13.951116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] PATH_INFO: `/v3/projects/d725470b8fbf40d6b50dc10dc7d47878` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:13.952311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d725470b8fbf40d6b50dc10dc7d47878)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:13.954256 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.000486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd725470b8fbf40d6b50dc10dc7d47878', 'request_id': 'req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:14.007233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd725470b8fbf40d6b50dc10dc7d47878', 'request_id': 'req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:14.012850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd725470b8fbf40d6b50dc10dc7d47878', 'request_id': 'req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:14.026269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] The token cache is being invalidate because project d725470b8fbf40d6b50dc10dc7d47878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:14.026588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:14.027229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-cd52d3bf-ae5f-4ce1-8a4b-a8f01ed9452a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:14.028803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1284/2589] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:13 2022] DELETE /identity/v3/projects/d725470b8fbf40d6b50dc10dc7d47878 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:14.033972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.034308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.034618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.053068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:14.054558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:14.120403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1306/2590] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:14.128200 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.128490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.128773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.147471 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:14.149303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:14.157201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1285/2591] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:14.161427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.171221 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2c51a-7015-486b-9fb7-0af43418ad46 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.175828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.176972 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.177255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.177529 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.178513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:26:14.178835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.180439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.181952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:14.195248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1307/2592] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:26:14.199474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.209128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89587c-a263-4410-b881-0a6597eb87fc None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.213985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.214697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.214985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.215260 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.215956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.217889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.245055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1286/2593] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:14 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:14.249047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.260555 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae14bce-2b65-4177-ba82-a0a3ac3c29e3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.318233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.318949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.319238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.319504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.320193 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.322113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.343099 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1308/2594] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:14 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:14.346944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.357526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b5b5-c395-49ce-bc40-6c82bd9133fd admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.362475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.363247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.363400 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.364166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.364941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.366562 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.380577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1287/2595] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:14.381888 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.392690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f5555-ce65-45e5-89d4-a24aba5e4ea8 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.397378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.398035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.398324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.398636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.399360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.400958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1bea-04af-436c-8716-4d5b988edff4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.414447 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1309/2596] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:14.415567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.425808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5df65-1f99-4ea7-964a-30c8ba0a5cb4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.431300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.432173 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.432448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.432719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] PATH_INFO: `/v3/projects/a14f8d2a8bce4836a64e34063fa8b869/users/e33f05465e4e44b7bac93a8dfb4b5868/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.447648 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14f8d2a8bce4836a64e34063fa8b869, user_id=e33f05465e4e44b7bac93a8dfb4b5868, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.450285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.463621 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1288/2597] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:14 2022] PUT /identity/v3/projects/a14f8d2a8bce4836a64e34063fa8b869/users/e33f05465e4e44b7bac93a8dfb4b5868/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:14.474780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.475076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.475357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.501696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] MFA Rules not processed for user `e33f05465e4e44b7bac93a8dfb4b5868`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:14.502955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:14.565900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1310/2598] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:14.576522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:14.578235 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:14.611329 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:14.638309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449e42-ff29-415b-9754-933fc5428202 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:14.642007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:14.642687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:14.642964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:14.643242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:14.651676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:14.653113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a9795-817c-4b6d-8181-7e2824402502 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:14.662633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1289/2599] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:21.859642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:21.921186 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a6097-f8ba-41dc-a147-4dcd31f5f8e6 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:21.926233 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:21.927126 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:21.927412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:21.927707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] PATH_INFO: `/v3/users/e33f05465e4e44b7bac93a8dfb4b5868` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:21.929160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33f05465e4e44b7bac93a8dfb4b5868)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:21.931128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:21.933647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:21.979487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33f05465e4e44b7bac93a8dfb4b5868'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:21.986709 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33f05465e4e44b7bac93a8dfb4b5868'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:21.987326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33f05465e4e44b7bac93a8dfb4b5868'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:21.999943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33f05465e4e44b7bac93a8dfb4b5868'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.000631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33f05465e4e44b7bac93a8dfb4b5868'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.014673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1311/2600] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:21 2022] DELETE /identity/v3/users/e33f05465e4e44b7bac93a8dfb4b5868 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:22.018952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.019406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.019785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.037788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:22.039036 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.098213 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1290/2601] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:22.107821 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.109227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.166884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4290b5-0f60-49df-9f0b-6555243e0236 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.170561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.171209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.171489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.171771 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.181703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.183180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5026455-b1c4-4f6b-ac3f-88667ddab28d service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.192945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1312/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:22.481324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.482812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.495018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0d09-9fd0-497f-81dc-79340c7e1b64 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.499710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.500573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.500862 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.501137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] PATH_INFO: `/v3/projects/a14f8d2a8bce4836a64e34063fa8b869` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.502488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14f8d2a8bce4836a64e34063fa8b869)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.504421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.551726 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14f8d2a8bce4836a64e34063fa8b869', 'request_id': 'req-97edac83-1b3e-4071-9b1c-b5532d44a2a6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.558004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14f8d2a8bce4836a64e34063fa8b869', 'request_id': 'req-97edac83-1b3e-4071-9b1c-b5532d44a2a6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.563511 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14f8d2a8bce4836a64e34063fa8b869', 'request_id': 'req-97edac83-1b3e-4071-9b1c-b5532d44a2a6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.576231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] The token cache is being invalidate because project a14f8d2a8bce4836a64e34063fa8b869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:22.576523 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.577181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-97edac83-1b3e-4071-9b1c-b5532d44a2a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:22.578789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1291/2603] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:22 2022] DELETE /identity/v3/projects/a14f8d2a8bce4836a64e34063fa8b869 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:22.584410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.584705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.584986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.605032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:22.606177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.675784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1313/2604] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:22.685049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.685343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.685624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.703588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:22.704795 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:22.713245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1292/2605] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:22.717441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.730675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5e59d-9d36-4156-b85d-34ef2d1f80bc None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.735222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.735868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.736148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.736436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.737132 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:26:22.737434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.739082 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.754442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1314/2606] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:26:22.755858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.765339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886ee6-55f3-4c9c-8dbb-57226b5780dd None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.771505 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.772204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.772489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.772766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.773458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.775414 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.804855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1293/2607] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:22.808579 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.819712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f56733-4287-411e-b38a-a4cf86d1876b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.828641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:22.883399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.884066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.884289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.884564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.885273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.887292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.909009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1315/2608] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:22 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:22.913082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.923906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109a6c9-7ace-4afa-86a2-22b88443aae0 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.928988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.929672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.929986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.930266 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.931039 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.932687 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.945243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1294/2609] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:22.949311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.960883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05ceb1-b315-4913-afd5-6391795d6774 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:22.965645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:22.966349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:22.966641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:22.966918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:22.967657 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:22.969263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac4d2a-4ac9-4196-a7dd-24e64867fb31 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:22.980368 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1316/2610] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:22.984879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:22.995592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3176-94a2-4859-bf95-771433faa8ca admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:23.000349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:23.001262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.001547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.001825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] PATH_INFO: `/v3/projects/cbbed76024fb401d9519d28dabbf6457/users/83e68d622a1c4e5892ea44b45257f133/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.015824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:23.019986 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbed76024fb401d9519d28dabbf6457, user_id=83e68d622a1c4e5892ea44b45257f133, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:23.025092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:23.038944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1295/2611] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:22 2022] PUT /identity/v3/projects/cbbed76024fb401d9519d28dabbf6457/users/83e68d622a1c4e5892ea44b45257f133/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:23.044365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35945-c9b6-48bb-9033-a47c5cda7886 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.044656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35945-c9b6-48bb-9033-a47c5cda7886 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.044936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c35945-c9b6-48bb-9033-a47c5cda7886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.068621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-40c35945-c9b6-48bb-9033-a47c5cda7886 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:23.069861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-40c35945-c9b6-48bb-9033-a47c5cda7886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:23.140531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1317/2612] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:23.151116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:23.152980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:23.228741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aba03-291c-4e60-8019-ca55a0bb8d16 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:23.234210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:23.235103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.235461 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.235879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.249815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:23.251800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:23.262890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1296/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:23.919125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdffab-f2ca-4b16-a6b3-85ce115f00eb None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:23.919987 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdffab-f2ca-4b16-a6b3-85ce115f00eb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:23.920417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdffab-f2ca-4b16-a6b3-85ce115f00eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:23.952183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-4bbdffab-f2ca-4b16-a6b3-85ce115f00eb None None] MFA Rules not processed for user `83e68d622a1c4e5892ea44b45257f133`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:23.961391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-4bbdffab-f2ca-4b16-a6b3-85ce115f00eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:24.055748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1318/2614] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:24.063692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:24.065402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:24.128271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39931-b304-4ae0-af30-fbc588be3293 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:24.131955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:24.132601 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:24.132883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:24.133157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:24.142189 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:24.143679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:24.153685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1297/2615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:28.648582 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:28.649364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:28.649364 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:28.667661 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] MFA Rules not processed for user `83e68d622a1c4e5892ea44b45257f133`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:28.668905 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:28.675688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1319/2616] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:28.713889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:28.715302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:28.790998 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73907ec7-0a3e-4348-a510-9661e718168b service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:28.794531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2861607f39d44172851da17c72704358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:28.795164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:28.795447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:28.795723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:28.797481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:28.827478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:28.867166 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:28.868599 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:28.879911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1298/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:29.078110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:29.079542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:29.091896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5711-2ce9-4057-a99f-c7f7f45b824d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:29.095469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:29.096097 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:29.096394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:29.096652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:29.098395 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:29.107506 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:29.108921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:29.113487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:29.117531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1320/2618] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:29.441778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:29.443645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:29.452928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:29.462679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd607450-cfaa-42b5-9138-a4feb1b82d27 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:29.466945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:29.467694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:29.468098 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:29.468513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:29.470791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:29.487108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:29.489996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:29.508104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1321/2619] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:26:29.537431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb31758-c604-4ac7-a356-faec1ba5989e service glance] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:29.544270 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:29.545054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:29.545439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:29.545883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:29.548212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:29.561770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:29.563699 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:29.576358 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1299/2620] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:35.179783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:35.190921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5db7ac-00bf-47e4-9b6d-319306e7dda4 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:35.195604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:35.196420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:35.196864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:35.197145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:35.199561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:35.212176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:35.214222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:35.227628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1322/2621] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:26:36.005075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:36.006958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:36.027922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2912a-fab1-4a68-b0b5-c86c76f4140b service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:36.034526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:36.035401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:36.035784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:36.036642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:36.038418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:36.053604 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:36.055502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:36.068945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1300/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:36.107933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:36.118756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d68de-7471-4f9d-a7a2-672d2bf67231 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:36.122678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:36.123326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:36.123611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:36.123886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:36.125800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:36.141346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:36.142967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:36.158813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1323/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:46.062814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=57 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:46.148629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=18 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:46.197407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:46.199090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:46.220101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841109-fbbc-47c9-bde2-d13f3926ed1a service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:46.235445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:46.236162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:46.236547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:46.237510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:46.240867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:46.260142 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:46.263368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b0f97-3a3d-4018-93b9-5949fe024ff2 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:46.281058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1301/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:46.718479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:46.720410 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:46.748151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1476495-3f64-46bb-99bc-5ae280aa4529 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:46.755854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:46.756796 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:46.757110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:46.757702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] PATH_INFO: `/v3/users/419a0c0359014f0e9f2226516e60ee50` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:46.759411 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=419a0c0359014f0e9f2226516e60ee50)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:46.763882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:46.832367 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419a0c0359014f0e9f2226516e60ee50'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:46.841360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419a0c0359014f0e9f2226516e60ee50'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:46.842429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419a0c0359014f0e9f2226516e60ee50'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:46.863884 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419a0c0359014f0e9f2226516e60ee50'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:46.865026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419a0c0359014f0e9f2226516e60ee50'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:46.879817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1324/2625] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:46 2022] DELETE /identity/v3/users/419a0c0359014f0e9f2226516e60ee50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:46.886595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:46.886884 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:46.887165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:46.906666 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:46.908043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:46.973997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:46.975764 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:47.028153 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1302/2626] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:47.041078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:47.042641 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:47.073394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85670d1e-8f43-426c-9be7-c9281d864a02 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:47.082371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:47.083416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:47.083768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:47.084155 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:47.086730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:47.106207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea48f26-b54a-408f-9196-c346c6775c4a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:47.110214 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:47.110787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:47.110964 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:47.111272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:47.111530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:47.116248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:47.121942 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:47.123587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:47.134645 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1325/2627] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:26:47.134811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1303/2628] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:26:47.465689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:47.468338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:47.485088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37e809-f840-48dc-b4a1-c31cc2a3915c service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:47.490502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:47.491489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:47.491810 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:47.492104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] PATH_INFO: `/v3/projects/9a48799b3c514bb7a154029b8cf29ab7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:47.493535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a48799b3c514bb7a154029b8cf29ab7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:47.496092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:47.554077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a48799b3c514bb7a154029b8cf29ab7', 'request_id': 'req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.561473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a48799b3c514bb7a154029b8cf29ab7', 'request_id': 'req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.568089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a48799b3c514bb7a154029b8cf29ab7', 'request_id': 'req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.582363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] The token cache is being invalidate because project 9a48799b3c514bb7a154029b8cf29ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:47.582711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.583560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.585339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1304/2629] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:47 2022] DELETE /identity/v3/projects/9a48799b3c514bb7a154029b8cf29ab7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:47.589685 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:47.600848 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa60f4-6998-4a7f-bc3d-864d5cb6855d service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:47.664297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:47.672609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:47.674220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:47.675236 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:47.675962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] PATH_INFO: `/v3/users/014999ad010a4df081f8ea5b4196caf6` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:47.678300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014999ad010a4df081f8ea5b4196caf6)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:47.680908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:47.758375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014999ad010a4df081f8ea5b4196caf6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.768573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014999ad010a4df081f8ea5b4196caf6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.769714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014999ad010a4df081f8ea5b4196caf6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.795029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014999ad010a4df081f8ea5b4196caf6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.795968 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8c7b51e6-152f-4174-a752-2147d49c5770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014999ad010a4df081f8ea5b4196caf6'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:47.831294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1326/2630] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:47 2022] DELETE /identity/v3/users/014999ad010a4df081f8ea5b4196caf6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:48.365425 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.366908 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:48.388578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff193f4b-27c6-4e94-8e60-9f3444cf86e7 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.400672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:48.454129 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.455031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.455366 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.456254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] PATH_INFO: `/v3/projects/1f72d1860d4e44c7b85754fafe22ba46` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.457760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f72d1860d4e44c7b85754fafe22ba46)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.460956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.514622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f72d1860d4e44c7b85754fafe22ba46', 'request_id': 'req-80d44b44-b90b-4269-ae2e-11c0a590180e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:48.524596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f72d1860d4e44c7b85754fafe22ba46', 'request_id': 'req-80d44b44-b90b-4269-ae2e-11c0a590180e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:48.533130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f72d1860d4e44c7b85754fafe22ba46', 'request_id': 'req-80d44b44-b90b-4269-ae2e-11c0a590180e'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:48.547817 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] The token cache is being invalidate because project 1f72d1860d4e44c7b85754fafe22ba46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:48.548105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:48.548751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-80d44b44-b90b-4269-ae2e-11c0a590180e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:48.550367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1305/2631] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:48 2022] DELETE /identity/v3/projects/1f72d1860d4e44c7b85754fafe22ba46 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:48.556177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.556570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.556931 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.577578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:48.578863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:48.651527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1327/2632] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:48.660851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.661431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.662193 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.682645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:48.683958 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:48.693727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1306/2633] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:26:48.698258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.710435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48cda0-fdbb-4730-8fc5-294a1fe53c6d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.715856 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.716756 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.717053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.717432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.718456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:26:48.718732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.720713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.735235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1328/2634] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:26:48.740099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.749744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79a848-fa0f-4c42-8c17-aeb25ce8dba4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.755205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.756100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.756389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.756694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.757375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.759348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.789766 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1307/2635] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:48 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:48.794492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.805997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b7e16-d00f-4b86-a689-71b116b31664 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.868935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.869731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.870029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.870422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.871128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.873205 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.883002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:48.899578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1329/2636] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:48 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:26:48.904888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.919615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65b20e-7d02-42f9-939e-23298f453e9c admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.926708 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.928011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.928319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.928600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.929453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.931383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:48.951865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1308/2637] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:48.953124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:48.975190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b28e9-f7a8-414a-9f93-ee831d653f02 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:48.982646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:48.983449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:48.983802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:48.984114 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:48.985101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:48.988230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e773c-80c0-4acc-9e4c-b3579028f547 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.009311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1330/2638] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:49.013347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.025339 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13443951-7945-42db-8184-bcefb7f53649 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.030518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.031409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.031696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.031975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] PATH_INFO: `/v3/projects/c674670f2eff40459fdba3dac5b738b6/users/e2945760b68b4c31a1beefcc2c690d92/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.048068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c674670f2eff40459fdba3dac5b738b6, user_id=e2945760b68b4c31a1beefcc2c690d92, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.051955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.067779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1309/2639] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:49 2022] PUT /identity/v3/projects/c674670f2eff40459fdba3dac5b738b6/users/e2945760b68b4c31a1beefcc2c690d92/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:49.080230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.080549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.080887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.110006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] MFA Rules not processed for user `e2945760b68b4c31a1beefcc2c690d92`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:49.111272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:49.178832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1331/2640] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:49.183873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.228695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:49.253591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbb916-1292-44bb-84b3-80b9477c1f7f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.258748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.259460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.259879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.260192 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.260897 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.263475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.301479 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1310/2641] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:49.305899 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.319654 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c8f9b-56e9-44fe-ad18-b119d2bb79e3 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.399324 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.401475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.401935 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.402686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.404632 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.410129 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.441093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1332/2642] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:26:49.447886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.473252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93fcc46-3b26-4743-ab5e-0fb7735edca3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.480655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.481341 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.481590 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.481898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.482803 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.484595 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.498080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1311/2643] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:49.502807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.514845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1243d-805d-417f-a5e0-dfd746f5d4b3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.520880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.521548 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.521835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.522115 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.522874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.524482 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4caf68-91bb-4b86-9716-a320663e81d4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.536951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1333/2644] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:49.541351 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.551730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9170c-8adf-4ec2-8baa-5dce62003fa4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.557143 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.558250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.558667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.559041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] PATH_INFO: `/v3/projects/308e70b9375f4c1e8ace990439192b42/users/871f1bda2fa049f0b32af89f35a0468b/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.577111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308e70b9375f4c1e8ace990439192b42, user_id=871f1bda2fa049f0b32af89f35a0468b, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.579830 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.593740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1312/2645] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:49 2022] PUT /identity/v3/projects/308e70b9375f4c1e8ace990439192b42/users/871f1bda2fa049f0b32af89f35a0468b/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:49.610886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.611182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.611457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.639861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] MFA Rules not processed for user `871f1bda2fa049f0b32af89f35a0468b`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:49.641280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:49.704882 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:49.737801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1334/2646] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:49.742689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.808096 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3335e-723d-47ff-a078-4345ff3b88f5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.812806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.813463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.813739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.814010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.815389 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.817289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.845204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1313/2647] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:26:49.849428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.861262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbb646-90ef-4809-9495-36434e424606 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.926414 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.927087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.927371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.927928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.928656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.930717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.954989 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1335/2648] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:26:49 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:26:49.960448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:49.972478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df1f0c-dda9-4d9c-b246-bef42db5ea28 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:49.977612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:49.978276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:49.978587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:49.978860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:49.979582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:49.981183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:49.994011 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1314/2649] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:49.998691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:50.012668 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09bc55-6530-4457-a578-d5d262c9fd70 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:50.017591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:50.019063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:50.019340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:50.019610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] PATH_INFO: `/v3/projects/f7248624b33a46d78b0717543ccae40d/users/e1025b792997407ba55be9450484c6d9/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:50.035339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7248624b33a46d78b0717543ccae40d, user_id=e1025b792997407ba55be9450484c6d9, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:50.038003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:50.055598 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1336/2650] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:49 2022] PUT /identity/v3/projects/f7248624b33a46d78b0717543ccae40d/users/e1025b792997407ba55be9450484c6d9/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:50.059655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:50.070563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb9e50-3fdb-4fc3-aaab-0186703b314d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:50.151079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:50.151762 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:50.152191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:50.152481 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:50.153212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:50.155066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c4d5c-153f-4ac4-8490-b157247b1b11 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:50.161447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:50.166933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1315/2651] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:26:50.171390 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:50.182166 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab91084-fa7f-42d3-a59e-06e13960b943 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:50.186924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:50.187822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:50.188107 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:50.188384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] PATH_INFO: `/v3/projects/f7248624b33a46d78b0717543ccae40d/users/e1025b792997407ba55be9450484c6d9/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:50.191011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7248624b33a46d78b0717543ccae40d, user_id=e1025b792997407ba55be9450484c6d9, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:50.193690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:50.205052 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1337/2652] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:26:50 2022] PUT /identity/v3/projects/f7248624b33a46d78b0717543ccae40d/users/e1025b792997407ba55be9450484c6d9/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:50.215092 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:50.215384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:50.215664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:50.248997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] MFA Rules not processed for user `e1025b792997407ba55be9450484c6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:50.250230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:50.323583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1316/2653] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:50.334868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:50.336864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:50.401778 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf73ccd-70e2-4f25-bad9-f679d70b28f3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:50.405533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:50.406183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:50.406503 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:50.406776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:50.418615 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:50.421180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:50.439863 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1338/2654] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:51.046932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:51.057644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250948c-8538-443d-87d2-84815b82aa36 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:51.061950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:51.062693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:51.063133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:51.063522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:51.139089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:51.141296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baa063-818f-499e-bb13-15e378d403cc service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:51.152802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1317/2655] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:55.459003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:55.470572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeb76-8e5c-43a8-a53a-993312e3bc81 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:55.474989 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:55.475679 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:55.476007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:55.476288 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:55.522696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:55.558982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:55.560470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:55.571722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1339/2656] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:56.305239 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7249-f4ab-45b6-be6d-c1737b2404dc None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:56.305999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7249-f4ab-45b6-be6d-c1737b2404dc None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:56.305999 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae7249-f4ab-45b6-be6d-c1737b2404dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:56.327071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-caae7249-f4ab-45b6-be6d-c1737b2404dc None None] MFA Rules not processed for user `83e68d622a1c4e5892ea44b45257f133`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:56.328579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-caae7249-f4ab-45b6-be6d-c1737b2404dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:56.406353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1318/2657] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:57.960665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:58.033133 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496355e-6221-4f34-8f91-d5b982414d16 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:58.038958 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:58.039900 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:58.040272 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:58.040614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] PATH_INFO: `/v3/users/e2945760b68b4c31a1beefcc2c690d92` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:58.041940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2945760b68b4c31a1beefcc2c690d92)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:58.044071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:58.097136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2945760b68b4c31a1beefcc2c690d92'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:58.106755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2945760b68b4c31a1beefcc2c690d92'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:58.110484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2945760b68b4c31a1beefcc2c690d92'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:58.125453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2945760b68b4c31a1beefcc2c690d92'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:58.126199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2945760b68b4c31a1beefcc2c690d92'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:58.141526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1340/2658] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:57 2022] DELETE /identity/v3/users/e2945760b68b4c31a1beefcc2c690d92 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:26:58.147532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:58.148465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:58.149082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:58.172500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:26:58.174325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:58.208672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:58.265559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1319/2659] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:26:58.279537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:58.281078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:58.371350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c0d18-14e9-4762-b3f1-abd24e7a608c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:58.378077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:58.379222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:58.379728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:58.380212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:58.394257 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:58.402491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:58.405428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:58.418294 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1341/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:26:59.030623 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:59.032627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:59.062859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642252e1-b36c-48f3-93ef-2148792c54b0 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:59.069107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:59.070540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:59.071655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:59.072137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] PATH_INFO: `/v3/projects/c674670f2eff40459fdba3dac5b738b6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:59.074439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c674670f2eff40459fdba3dac5b738b6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:59.076950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:59.147514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c674670f2eff40459fdba3dac5b738b6', 'request_id': 'req-e5233323-fe89-48f7-82fe-060c1b4a21b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.155162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c674670f2eff40459fdba3dac5b738b6', 'request_id': 'req-e5233323-fe89-48f7-82fe-060c1b4a21b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.163770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c674670f2eff40459fdba3dac5b738b6', 'request_id': 'req-e5233323-fe89-48f7-82fe-060c1b4a21b4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.181418 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] The token cache is being invalidate because project c674670f2eff40459fdba3dac5b738b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:59.181813 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.182801 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.184664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1320/2661] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:59 2022] DELETE /identity/v3/projects/c674670f2eff40459fdba3dac5b738b6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:59.188881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:59.200110 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f641569-3e56-4615-b320-151ad2a5fec9 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:59.283874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:59.285101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:59.285696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:59.286811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] PATH_INFO: `/v3/users/871f1bda2fa049f0b32af89f35a0468b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:59.288465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=871f1bda2fa049f0b32af89f35a0468b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:59.290861 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:59.391827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871f1bda2fa049f0b32af89f35a0468b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.406580 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871f1bda2fa049f0b32af89f35a0468b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.407670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871f1bda2fa049f0b32af89f35a0468b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.435454 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871f1bda2fa049f0b32af89f35a0468b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.436793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871f1bda2fa049f0b32af89f35a0468b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.454384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1342/2662] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:59 2022] DELETE /identity/v3/users/871f1bda2fa049f0b32af89f35a0468b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:59.772693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:59.774519 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:59.790032 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233323-fe89-48f7-82fe-060c1b4a21b4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:26:59.852440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:26:59.853303 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:26:59.853580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:26:59.853850 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] PATH_INFO: `/v3/projects/308e70b9375f4c1e8ace990439192b42` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:26:59.855031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=308e70b9375f4c1e8ace990439192b42)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:26:59.856941 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:26:59.904571 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308e70b9375f4c1e8ace990439192b42', 'request_id': 'req-f87945a9-c853-4b0b-9560-67d0af9ebdbe'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.910797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308e70b9375f4c1e8ace990439192b42', 'request_id': 'req-f87945a9-c853-4b0b-9560-67d0af9ebdbe'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.912006 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:26:59.917673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308e70b9375f4c1e8ace990439192b42', 'request_id': 'req-f87945a9-c853-4b0b-9560-67d0af9ebdbe'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.935123 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] The token cache is being invalidate because project 308e70b9375f4c1e8ace990439192b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:26:59.935474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.936162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:26:59.938917 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1321/2663] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:26:59 2022] DELETE /identity/v3/projects/308e70b9375f4c1e8ace990439192b42 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:26:59.944585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:26:59.948238 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:26:59.976702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b660edc-1143-4ece-8890-93daf61bf33e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.059862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.060785 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.061070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.061353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] PATH_INFO: `/v3/users/e1025b792997407ba55be9450484c6d9` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.064156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1025b792997407ba55be9450484c6d9)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:00.066745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:00.091641 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:00.132538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1025b792997407ba55be9450484c6d9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.143453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1025b792997407ba55be9450484c6d9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.144218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1025b792997407ba55be9450484c6d9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.161647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1025b792997407ba55be9450484c6d9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.162486 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-459d554d-487f-4ee2-999d-0fd311c7960b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1025b792997407ba55be9450484c6d9'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.175616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1343/2664] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:26:59 2022] DELETE /identity/v3/users/e1025b792997407ba55be9450484c6d9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:00.470054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:00.471731 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:00.485647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87945a9-c853-4b0b-9560-67d0af9ebdbe admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.545367 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.546427 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.546781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.547098 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] PATH_INFO: `/v3/projects/f7248624b33a46d78b0717543ccae40d` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.548576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7248624b33a46d78b0717543ccae40d)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:00.550543 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:00.604272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7248624b33a46d78b0717543ccae40d', 'request_id': 'req-84f7bedd-982a-4b7c-8c81-d521cb65db63'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.613828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7248624b33a46d78b0717543ccae40d', 'request_id': 'req-84f7bedd-982a-4b7c-8c81-d521cb65db63'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.623407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7248624b33a46d78b0717543ccae40d', 'request_id': 'req-84f7bedd-982a-4b7c-8c81-d521cb65db63'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.642363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] The token cache is being invalidate because project f7248624b33a46d78b0717543ccae40d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:00.642703 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.643333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-84f7bedd-982a-4b7c-8c81-d521cb65db63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:00.649716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1322/2665] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:00 2022] DELETE /identity/v3/projects/f7248624b33a46d78b0717543ccae40d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:00.651352 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.651865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.651959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.671667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:00.672889 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:00.742930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1344/2666] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:00.744758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.745057 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.745392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.764871 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:00.766125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:00.774168 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1323/2667] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:00.778711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:00.788850 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e319-1c3f-4d64-850b-661fcf1a2d4c None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.795892 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.796560 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.796841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.797113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.797828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:27:00.798130 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:00.802759 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:00.816830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1345/2668] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:27:00.821202 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:00.830858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0afd1-c9f8-49ac-87d7-989083881fd4 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.836041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.836700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.836980 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.837257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.837957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:00.839893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:00.866467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1324/2669] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:00.870623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:00.881428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a534c2-236c-407d-a39b-e2123c2c1086 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.939128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.940383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:00.940667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:00.940933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:00.941697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:00.945467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:00.976828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1346/2670] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:27:00 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:27:00.981541 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:00.993789 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56993f18-cec4-41f7-a530-b8c4c966d9bf admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:00.999062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:00.999776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.000137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.000588 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.001364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.003760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.015826 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1325/2671] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:01.020885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.032101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ef89-4e11-45b0-a67b-71fb39134c73 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.037817 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.038553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.038836 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.039116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.039841 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.041536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf518ba8-a510-471a-98cc-d5219ed13cf1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.047830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:01.055359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1347/2672] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:01.059619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.070530 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3abc6-6f94-49cf-b42f-15129a802995 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.075739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.076670 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.076953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.077236 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] PATH_INFO: `/v3/projects/5ea22aff6927417e934bcfeea252e410/users/4828261dd5d94e8b91fc43c1e61db174/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.097411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea22aff6927417e934bcfeea252e410, user_id=4828261dd5d94e8b91fc43c1e61db174, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.100274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.134560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1326/2673] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:01 2022] PUT /identity/v3/projects/5ea22aff6927417e934bcfeea252e410/users/4828261dd5d94e8b91fc43c1e61db174/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:01.143426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.143877 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.146357 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.177628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] MFA Rules not processed for user `4828261dd5d94e8b91fc43c1e61db174`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:01.178897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:01.253854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1348/2674] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:01.258956 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.270421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:01.336080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c44d6-9c8b-453a-8a30-4282eb4258be admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.341034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.341628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.341912 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.342186 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.342922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.344879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.372417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1327/2675] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:27:01 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:01.377160 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.390890 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aec615-8b64-4756-93c7-cdcc8a27e042 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.474275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.475356 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.475768 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.476175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.477125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.479718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.509649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1349/2676] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:27:01 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:01.511181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.523328 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0660d-9e6c-49d7-9882-00dc17b64fbe admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.528911 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.529580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.529855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.530152 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.530906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.532951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.544692 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1328/2677] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:01.549040 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.559508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb893fa0-62a9-4caa-afe4-68d40f93ee94 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.564194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.565006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.565321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.565597 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] PATH_INFO: `/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3/users/62175ca7374e4fbdb17a523b38b93c95/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.583932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94b91e51de643c7bbbe2d6d1e0f32e3, user_id=62175ca7374e4fbdb17a523b38b93c95, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.586787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.608674 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1350/2678] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:01 2022] PUT /identity/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3/users/62175ca7374e4fbdb17a523b38b93c95/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:01.608825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.615546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.617137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:01.620663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff91ce-178b-49ef-8f4c-e2fd2bd5b495 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.678057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf7230-7546-4145-83ef-b7b2c0be95fa admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.680114 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.680793 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.681078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.681359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.681947 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.682077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.682613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.682896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.683168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.683700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.684945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:01.694209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1329/2679] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:01.698493 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.700131 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:01.708988 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7847b-5282-4811-816b-0f6d11b68413 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:01.713843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:01.714758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.715046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.715320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] PATH_INFO: `/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3/users/62175ca7374e4fbdb17a523b38b93c95/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.717466 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94b91e51de643c7bbbe2d6d1e0f32e3, user_id=62175ca7374e4fbdb17a523b38b93c95, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.720169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3e6cf-a8ea-41f0-9553-406b08a3d848 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.737004 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1330/2680] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:01 2022] PUT /identity/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3/users/62175ca7374e4fbdb17a523b38b93c95/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:27:01.747091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:01.747397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:01.747647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:01.770346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:01.771838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:01.777016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] MFA Rules not processed for user `62175ca7374e4fbdb17a523b38b93c95`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:01.778240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:01.782299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1351/2681] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:27:01.834785 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:01.855021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1331/2682] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:01.867317 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:01.869248 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:01.932140 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97129de-979c-43bc-9a2a-5386f4fef79e service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:02.007867 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:02.008601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:02.008891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:02.009265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:02.082429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:02.083883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:02.093437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1352/2683] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:06.344338 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:06.345857 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:06.380344 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d44b-3bf1-40cd-9dcd-8f88838c2cf1 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:06.383990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:06.384645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:06.384927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:06.385310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:06.420138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:06.421938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304c008-afd9-4012-901e-7ca897e87eb9 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:06.432780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1332/2684] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:11.305805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:11.323967 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:11.390240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2bd8-49d7-4ad0-8908-2465eb392867 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:11.395274 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:11.396734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:11.397022 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:11.397305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] PATH_INFO: `/v3/users/4828261dd5d94e8b91fc43c1e61db174` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:11.398565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4828261dd5d94e8b91fc43c1e61db174)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:11.400557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:11.452087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4828261dd5d94e8b91fc43c1e61db174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:11.459603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4828261dd5d94e8b91fc43c1e61db174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:11.460255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4828261dd5d94e8b91fc43c1e61db174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:11.472807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4828261dd5d94e8b91fc43c1e61db174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:11.473480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4828261dd5d94e8b91fc43c1e61db174'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:11.485479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1353/2685] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:27:11 2022] DELETE /identity/v3/users/4828261dd5d94e8b91fc43c1e61db174 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:11.490540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:11.491070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:11.491165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:11.508431 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:11.509761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:11.568446 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1333/2686] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:11.578062 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:11.579492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:11.639389 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8145ab-e5d9-4ecf-9c94-cfe5cda4c3eb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:11.643138 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:11.643782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:11.644066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:11.644341 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:11.653959 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:11.655424 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:11.665995 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1354/2687] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:12.025031 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:12.026822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:12.039561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca7eba-939c-450d-80d6-6bb932e45430 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:12.044749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:12.045682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:12.045985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:12.046271 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] PATH_INFO: `/v3/projects/5ea22aff6927417e934bcfeea252e410` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:12.048788 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea22aff6927417e934bcfeea252e410)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:12.050841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:12.117970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea22aff6927417e934bcfeea252e410', 'request_id': 'req-aefe3887-772e-43f6-8d57-1b55b8a2ad57'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.124457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea22aff6927417e934bcfeea252e410', 'request_id': 'req-aefe3887-772e-43f6-8d57-1b55b8a2ad57'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.132410 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea22aff6927417e934bcfeea252e410', 'request_id': 'req-aefe3887-772e-43f6-8d57-1b55b8a2ad57'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.150169 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] The token cache is being invalidate because project 5ea22aff6927417e934bcfeea252e410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:12.150488 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.151136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.152700 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1334/2688] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:12 2022] DELETE /identity/v3/projects/5ea22aff6927417e934bcfeea252e410 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:27:12.156702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:12.167378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e09d0-2d17-4fcb-aeaf-e71c9b4378ee service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:12.190820 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:12.239770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:12.239770 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:12.240149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:12.240296 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] PATH_INFO: `/v3/users/62175ca7374e4fbdb17a523b38b93c95` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:12.241860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62175ca7374e4fbdb17a523b38b93c95)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:12.243922 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:12.309739 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62175ca7374e4fbdb17a523b38b93c95'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.320546 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62175ca7374e4fbdb17a523b38b93c95'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.321302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62175ca7374e4fbdb17a523b38b93c95'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.335719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62175ca7374e4fbdb17a523b38b93c95'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.336542 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62175ca7374e4fbdb17a523b38b93c95'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.355247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1355/2689] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:27:12 2022] DELETE /identity/v3/users/62175ca7374e4fbdb17a523b38b93c95 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:12.558730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:12.560300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:12.593542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:12.664542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe3887-772e-43f6-8d57-1b55b8a2ad57 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:12.717549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:12.720599 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:12.738855 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:12.739522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:12.739808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:12.740087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:12.741914 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:12.754582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:12.756215 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965282a8-aba4-4f44-be2c-91649ffc07f5 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:12.767565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1335/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:12.803285 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d67f84-60c2-4f42-8d30-d539db228a75 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:12.807825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:12.808690 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:12.809676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:12.809676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] PATH_INFO: `/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:12.810711 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94b91e51de643c7bbbe2d6d1e0f32e3)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:12.812629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:12.858376 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:12.862527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94b91e51de643c7bbbe2d6d1e0f32e3', 'request_id': 'req-289d42cc-cc43-4964-bc45-7646eece6a94'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.870437 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94b91e51de643c7bbbe2d6d1e0f32e3', 'request_id': 'req-289d42cc-cc43-4964-bc45-7646eece6a94'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.877633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94b91e51de643c7bbbe2d6d1e0f32e3', 'request_id': 'req-289d42cc-cc43-4964-bc45-7646eece6a94'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.890366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] The token cache is being invalidate because project c94b91e51de643c7bbbe2d6d1e0f32e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:12.890659 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.891545 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-289d42cc-cc43-4964-bc45-7646eece6a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:12.893400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1356/2691] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:12 2022] DELETE /identity/v3/projects/c94b91e51de643c7bbbe2d6d1e0f32e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:12.900946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:12.901279 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:12.901509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:12.921661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:12.923043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:12.994374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1336/2692] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:13.004027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.004323 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.004610 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.024171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:13.025453 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:13.034182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1357/2693] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:13.038535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.048473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699afada-dcac-4560-a863-dfd451d11a13 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.053272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.053915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.054197 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.054513 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.055228 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:27:13.055532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.057120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.068402 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1337/2694] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:27:13.072541 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.082106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef247f-cc2e-4de4-a3e2-2398b720205b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.086748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.087391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.087663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.087933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.088619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.090533 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.116000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1358/2695] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:27:13 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:13.120070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.129981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de2d8-868d-42ce-9d77-37523799e137 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.150091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:13.188273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.188946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.189232 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.189510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.190204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.192180 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.214719 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1338/2696] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:27:13 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:13.219173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.230829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aef624-84aa-4062-a095-6d3e58714a77 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.235865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.236630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.237673 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.237957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.238925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.240748 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.253275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1359/2697] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:13.257383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.268040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08845dee-c861-49ea-a276-5d4639f15757 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.272814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.273489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.273776 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.274055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.274822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.276445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb78a6f-46b6-4243-9e13-2180ad36af6f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.288077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1339/2698] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:13.292103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.302755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a11063-111f-498f-9681-bd5311dd6767 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.307627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.308513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.308796 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.309072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] PATH_INFO: `/v3/projects/ba18ba6cf0254f51b04dc9cf6200695b/users/64cbd4169a2a48c3ba8960c1badcdd08/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.324467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba18ba6cf0254f51b04dc9cf6200695b, user_id=64cbd4169a2a48c3ba8960c1badcdd08, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.327304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.342428 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1360/2699] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:13 2022] PUT /identity/v3/projects/ba18ba6cf0254f51b04dc9cf6200695b/users/64cbd4169a2a48c3ba8960c1badcdd08/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:13.355065 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db6327-c887-46ed-93b3-83f9a6aca174 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.355483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db6327-c887-46ed-93b3-83f9a6aca174 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.355839 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db6327-c887-46ed-93b3-83f9a6aca174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.383353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-e3db6327-c887-46ed-93b3-83f9a6aca174 None None] MFA Rules not processed for user `64cbd4169a2a48c3ba8960c1badcdd08`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:13.384502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-e3db6327-c887-46ed-93b3-83f9a6aca174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:13.448895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1340/2700] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:13.460835 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:13.462559 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:13.520564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8a8a8-d916-466d-a5f1-52f033c8d263 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:13.524174 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:13.524805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:13.525084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:13.525359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:13.534048 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:13.535505 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:13.544145 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1361/2701] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:23.445972 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:23.540345 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:26.059290 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a115-0785-450f-9b50-7cc9939b3657 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:26.059946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a115-0785-450f-9b50-7cc9939b3657 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:26.059946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12a115-0785-450f-9b50-7cc9939b3657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:26.138685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-8c12a115-0785-450f-9b50-7cc9939b3657 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:26.140174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8c12a115-0785-450f-9b50-7cc9939b3657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:26.223806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1341/2702] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 18:27:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:26.277478 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:26.279071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:26.386910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869404e5-59b1-4fa1-bcfa-efb471163efb service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:26.393412 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:26.394539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:26.394953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:26.395321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:26.406229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:26.409929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:26.424774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1362/2703] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:27:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:27:36.218725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=44 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:36.420948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=41 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:42.240183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:42.240720 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:42.241010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:42.259508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:42.260740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:42.268509 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1342/2704] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 18:27:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:42.282604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:42.293388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e83215-74ea-4fa7-9047-1fd29c44f54e service swift] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:42.298845 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:42.299496 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:42.299782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:42.300061 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:42.309337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:42.310954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba76ca-cf2c-407d-8394-6d815d026bbe service swift] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:42.319402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1363/2705] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 18:27:42.631650 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:42.727028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cbe8f-60a7-4d29-9352-d06ea8ea0cf6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:42.731875 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:42.732758 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:42.733044 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:42.733324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] PATH_INFO: `/v3/users/64cbd4169a2a48c3ba8960c1badcdd08` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:42.735209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64cbd4169a2a48c3ba8960c1badcdd08)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:42.737250 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:42.793560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64cbd4169a2a48c3ba8960c1badcdd08'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:42.794395 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64cbd4169a2a48c3ba8960c1badcdd08'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:42.807775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64cbd4169a2a48c3ba8960c1badcdd08'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:42.815783 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64cbd4169a2a48c3ba8960c1badcdd08'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:42.816559 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64cbd4169a2a48c3ba8960c1badcdd08'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:42.830082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1343/2706] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:27:42 2022] DELETE /identity/v3/users/64cbd4169a2a48c3ba8960c1badcdd08 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:42.834761 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:42.835071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:42.835350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:42.853911 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:42.855163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:42.920194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1364/2707] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:42.931639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:42.933365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:43.002473 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc82588-95fe-4820-acff-9949c7c53a6a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:43.008381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:43.009428 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:43.009975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:43.010540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:43.022591 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:43.024055 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:43.036570 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1344/2708] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:43.337573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:43.339195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:43.352846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78414f-732e-4775-895e-a22895aaa87b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:43.357608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:43.358518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:43.358837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:43.359117 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] PATH_INFO: `/v3/projects/ba18ba6cf0254f51b04dc9cf6200695b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:43.360329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba18ba6cf0254f51b04dc9cf6200695b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:43.362312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:43.414349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba18ba6cf0254f51b04dc9cf6200695b', 'request_id': 'req-5ab90537-096f-4dfb-ac6e-f8f35786d508'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:43.419782 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba18ba6cf0254f51b04dc9cf6200695b', 'request_id': 'req-5ab90537-096f-4dfb-ac6e-f8f35786d508'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:43.431084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba18ba6cf0254f51b04dc9cf6200695b', 'request_id': 'req-5ab90537-096f-4dfb-ac6e-f8f35786d508'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:43.443719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] The token cache is being invalidate because project ba18ba6cf0254f51b04dc9cf6200695b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:43.443719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:43.444124 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:43.446295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1365/2709] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:43 2022] DELETE /identity/v3/projects/ba18ba6cf0254f51b04dc9cf6200695b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:44.269885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:44.271355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:44.297824 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:44.334776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:44.336209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:44.362292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257dcd7-acbd-4272-ae01-08a3e8d0b3e1 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:44.412702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90537-096f-4dfb-ac6e-f8f35786d508 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:44.417417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:44.418293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:44.418627 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:44.418902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] PATH_INFO: `/v3/users/83e68d622a1c4e5892ea44b45257f133` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:44.420105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e68d622a1c4e5892ea44b45257f133)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:44.422633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:44.429463 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:44.431262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:44.431262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:44.431262 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:44.432930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:44.437434 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:44.453113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:44.456061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:44.474104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1345/2710] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:27:44.478975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e68d622a1c4e5892ea44b45257f133'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.489031 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e68d622a1c4e5892ea44b45257f133'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.489725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e68d622a1c4e5892ea44b45257f133'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.508651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e68d622a1c4e5892ea44b45257f133'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.509416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-adc84d23-a07b-4043-8a54-f968b127890f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e68d622a1c4e5892ea44b45257f133'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.523448 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1366/2711] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:27:44 2022] DELETE /identity/v3/users/83e68d622a1c4e5892ea44b45257f133 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:27:44.709797 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:44.723561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450a4b0-98b3-4fbc-b4d3-e30c3e25a047 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:44.785642 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:44.786535 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:44.786814 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:44.787081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] PATH_INFO: `/v3/projects/cbbed76024fb401d9519d28dabbf6457` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:44.788322 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbed76024fb401d9519d28dabbf6457)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:44.790240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:44.838275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbed76024fb401d9519d28dabbf6457', 'request_id': 'req-a9688f0b-bada-4269-8525-5ca33c81b935'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.840292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:44.846268 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbed76024fb401d9519d28dabbf6457', 'request_id': 'req-a9688f0b-bada-4269-8525-5ca33c81b935'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.853087 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbed76024fb401d9519d28dabbf6457', 'request_id': 'req-a9688f0b-bada-4269-8525-5ca33c81b935'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.867380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] The token cache is being invalidate because project cbbed76024fb401d9519d28dabbf6457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:44.867674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.868340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-a9688f0b-bada-4269-8525-5ca33c81b935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:44.869925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1346/2712] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:44 2022] DELETE /identity/v3/projects/cbbed76024fb401d9519d28dabbf6457 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:44.875652 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:44.876068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:44.876425 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:44.896078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:44.897333 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:44.979375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1367/2713] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:44.990010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:44.990442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:44.990779 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.016102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:45.017332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:45.025669 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1347/2714] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:45.030066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.040388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9137aa-0e77-4f8d-814a-970bd68c9cf8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.045407 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.046064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.046403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.046681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.047625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:27:45.047930 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.049608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.063059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1368/2715] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:27:45.067426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.076896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44760e7-088f-407f-b4f8-ca2ee33a2df6 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.082450 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.083105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.083385 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.083657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.084364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.086319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.112365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1348/2716] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:27:45 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:27:45.116501 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.126953 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095fe1b-5167-4355-a4be-5c0c81f65b52 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.192780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.193464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.193745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.194023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.194773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.196726 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.222781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1369/2717] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:27:45 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:45.226984 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.240021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d238b4-63f6-4bb3-ba37-1f6cb2013d1b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.244716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.245363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.245645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.245920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.246685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.248292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.260332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1349/2718] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:27:45.264336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.274743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab99464-ac40-4624-b1fc-457a9b199152 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.279421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.280073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.280351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.280701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.281924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.283564 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18b474-5037-4fe6-988f-ea14b7aa18f4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.284992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:45.295929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1370/2719] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:45.300165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.310289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca221b-49a9-4f51-b452-7cb937fa236a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.314823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.315694 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.315971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.316241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] PATH_INFO: `/v3/projects/7ad8732a34af4f4b940e2f520873e725/users/c00dd6a6e9f74b739cb8428736b4d20f/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.330967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad8732a34af4f4b940e2f520873e725, user_id=c00dd6a6e9f74b739cb8428736b4d20f, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.333607 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.346404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1350/2720] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:45 2022] PUT /identity/v3/projects/7ad8732a34af4f4b940e2f520873e725/users/c00dd6a6e9f74b739cb8428736b4d20f/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:27:45.356896 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.357183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.357455 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.383101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] MFA Rules not processed for user `c00dd6a6e9f74b739cb8428736b4d20f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:45.384227 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:45.447591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1371/2721] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:45.459302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.460965 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:45.519147 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4ad9c-7030-4855-8a41-18a25a181c7f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.522932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.523582 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.523865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.524145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.533394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.534874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883b32-c8a6-429d-b25b-2cc02c4dd06f service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.543611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1351/2722] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:45.572002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.645997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17061dd5-a7e2-4933-8b94-456144af7189 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.650751 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.651626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.651908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.652209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] PATH_INFO: `/v3/users/c00dd6a6e9f74b739cb8428736b4d20f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.653418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00dd6a6e9f74b739cb8428736b4d20f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.655398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.701498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00dd6a6e9f74b739cb8428736b4d20f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:45.708326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00dd6a6e9f74b739cb8428736b4d20f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:45.709084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00dd6a6e9f74b739cb8428736b4d20f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:45.721570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00dd6a6e9f74b739cb8428736b4d20f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:45.722222 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00dd6a6e9f74b739cb8428736b4d20f'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:45.734773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1372/2723] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:27:45 2022] DELETE /identity/v3/users/c00dd6a6e9f74b739cb8428736b4d20f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:45.739424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.739713 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.739990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.757086 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:45.758287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:45.821661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1352/2724] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:45.831312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:45.832717 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:45.839337 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:45.891920 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d339-4e0e-4aba-803e-f4a1bfb84491 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:45.895612 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:45.896313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:45.896552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:45.896832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:45.907201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:45.908640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafe6fe-2d2a-4b15-88e6-ce86ad6ce93b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:45.918594 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1373/2725] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:46.209375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.211105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:46.233042 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f12a-8de4-4479-930a-420742c0fa6e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.241160 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.242238 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.242626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.242960 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] PATH_INFO: `/v3/projects/7ad8732a34af4f4b940e2f520873e725` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.244310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad8732a34af4f4b940e2f520873e725)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.246823 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.260330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:46.299077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad8732a34af4f4b940e2f520873e725', 'request_id': 'req-428443bf-0702-4c5c-806b-3c190684c848'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:46.305684 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad8732a34af4f4b940e2f520873e725', 'request_id': 'req-428443bf-0702-4c5c-806b-3c190684c848'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:46.311286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad8732a34af4f4b940e2f520873e725', 'request_id': 'req-428443bf-0702-4c5c-806b-3c190684c848'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:46.325796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] The token cache is being invalidate because project 7ad8732a34af4f4b940e2f520873e725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:27:46.326088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:46.326751 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-428443bf-0702-4c5c-806b-3c190684c848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:27:46.328308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1353/2726] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:27:46 2022] DELETE /identity/v3/projects/7ad8732a34af4f4b940e2f520873e725 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:46.333286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.333601 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.333883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.352999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:46.354209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:46.415593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1374/2727] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:46.423761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.424103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.424368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.443275 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:46.444516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:46.452549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1354/2728] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:46.458143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.468857 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454375-6dcb-47d5-8d39-754563a38ee8 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.473553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.474245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.474566 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.474868 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.475596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:27:46.475901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.477513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.490535 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1375/2729] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:27:46.494891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.504368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb5a0e-5a2a-432a-b1bd-2b3568a8c79c None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.509636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.510298 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.510613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.510889 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.511589 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.513510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.540311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1355/2730] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:27:46 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:46.544527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.555924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d878c76-e0f4-47e6-a3a0-4e72f04b4bc3 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.618175 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.618740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.619053 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.619301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.620001 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.622046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.642681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1376/2731] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:27:46 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:27:46.647412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.659496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdab0d-167c-431f-a4e2-0016c5da03e8 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.665485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.666182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.666422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.666695 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.667444 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.669375 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.690043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1356/2732] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:27:46.694969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.707216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5973ba9-2be7-4af2-baef-204c07eed36a admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.711870 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.712536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.712826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.713105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.713828 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.715463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151f58-138a-4633-b6ef-388656ace761 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.726646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1377/2733] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:27:46.730848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.742518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b83a7-70dc-4a42-8888-252a2dc8c01a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.747157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.748029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.748310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.748583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] PATH_INFO: `/v3/projects/69923bd737bb4772852723c5cd99d57f/users/13343af4cf404f0ea75ebd87eeb4d291/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.769567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69923bd737bb4772852723c5cd99d57f, user_id=13343af4cf404f0ea75ebd87eeb4d291, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.772293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:46.787611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1357/2734] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:27:46 2022] PUT /identity/v3/projects/69923bd737bb4772852723c5cd99d57f/users/13343af4cf404f0ea75ebd87eeb4d291/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:27:46.798163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a627dc-031a-4883-86fe-e0c83d97ca6b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.798484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a627dc-031a-4883-86fe-e0c83d97ca6b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.798797 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a627dc-031a-4883-86fe-e0c83d97ca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.832951 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:46.836255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-01a627dc-031a-4883-86fe-e0c83d97ca6b None None] MFA Rules not processed for user `13343af4cf404f0ea75ebd87eeb4d291`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:46.838065 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-01a627dc-031a-4883-86fe-e0c83d97ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:46.904513 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1378/2735] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:27:46.918635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:46.920759 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:46.982284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576003e1-f25d-40ba-947a-884f617a326f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:46.985975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:46.986665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:46.986950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:46.987226 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:46.996626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:46.998077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:47.007840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1358/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:27:48.385730 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ad8a-8f63-4cd1-bd7a-937be88b6768 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:48.386387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ad8a-8f63-4cd1-bd7a-937be88b6768 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:48.386387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0ad8a-8f63-4cd1-bd7a-937be88b6768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:48.406957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-21f0ad8a-8f63-4cd1-bd7a-937be88b6768 None None] MFA Rules not processed for user `13343af4cf404f0ea75ebd87eeb4d291`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:27:48.408203 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-21f0ad8a-8f63-4cd1-bd7a-937be88b6768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:48.415325 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1379/2737] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:27:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:27:57.003394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:58.399439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=29 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:27:59.317314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:27:59.346209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39f109-1582-4a41-874f-ba400d86314f service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:27:59.350008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:27:59.350678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:27:59.350959 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:27:59.351233 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:27:59.353034 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:27:59.436181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:27:59.437617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:27:59.457660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1359/2738] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:27:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:03.813210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97647dd-1953-41b3-b85e-cb132a2de4b9 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:03.813719 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97647dd-1953-41b3-b85e-cb132a2de4b9 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:03.813819 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97647dd-1953-41b3-b85e-cb132a2de4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:03.831426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f97647dd-1953-41b3-b85e-cb132a2de4b9 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:03.832625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f97647dd-1953-41b3-b85e-cb132a2de4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:03.886851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1380/2739] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:03.898613 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:03.899982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:03.970230 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062778ec-7575-44c4-8df1-36d92c0fb06a service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:03.975151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:03.975791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:03.976066 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:03.976334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:03.984929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:03.987060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:03.995609 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1360/2740] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:28:04.546150 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ff64-3d94-4472-becb-1b1ccff687cb None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:04.546475 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ff64-3d94-4472-becb-1b1ccff687cb None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:04.546752 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ff64-3d94-4472-becb-1b1ccff687cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:04.569068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f8a5ff64-3d94-4472-becb-1b1ccff687cb None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:04.570303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f8a5ff64-3d94-4472-becb-1b1ccff687cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:04.577658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1381/2741] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 18:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:04.589711 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:04.601874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a6d3a-8af2-4808-b884-23afc43f7fd7 service swift] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:04.607148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:04.608365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:04.608657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:04.608933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:04.617979 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:04.619614 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:04.628384 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1361/2742] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:28:04.819169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae25e5-3d58-46b6-8466-661e38e5bd48 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:04.819660 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae25e5-3d58-46b6-8466-661e38e5bd48 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:04.819763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae25e5-3d58-46b6-8466-661e38e5bd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:04.837557 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-f4ae25e5-3d58-46b6-8466-661e38e5bd48 None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:04.838807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f4ae25e5-3d58-46b6-8466-661e38e5bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:04.846456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1382/2743] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jun 30 18:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:04.858306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:04.869054 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4e3e5-0200-401d-881b-1c1587432e05 service swift] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:04.874059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:04.874725 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:04.875007 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:04.875280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:04.883671 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:04.885243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:04.893953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1362/2744] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:28:05.225224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:05.225520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:05.225802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:05.243696 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] MFA Rules not processed for user `9425457963714156a39a1eced4c8f85f`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:05.244878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:05.251964 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1383/2745] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jun 30 18:28:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:05.264247 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:05.274728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9336c-22e1-4470-8a92-d433f15f49b1 service swift] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:05.280211 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ec0b4d695474375b2f87727d41e40f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:05.280843 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:05.281121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:05.281392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:05.289737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:05.291333 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268ebc0-1363-4907-9b9f-e8583c8bcc69 service swift] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:05.299943 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1363/2746] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jun 30 18:28:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 18:28:07.353350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:07.417492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61e8fa-d817-49e1-9c1a-4bdd1c2bf8af None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:07.422670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:07.423561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:07.423838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:07.424108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] PATH_INFO: `/v3/users/13343af4cf404f0ea75ebd87eeb4d291` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:07.425289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13343af4cf404f0ea75ebd87eeb4d291)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:07.427492 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:07.472638 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13343af4cf404f0ea75ebd87eeb4d291'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.479825 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13343af4cf404f0ea75ebd87eeb4d291'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.480449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13343af4cf404f0ea75ebd87eeb4d291'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.494767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13343af4cf404f0ea75ebd87eeb4d291'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.495463 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13343af4cf404f0ea75ebd87eeb4d291'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.507849 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1384/2747] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:07 2022] DELETE /identity/v3/users/13343af4cf404f0ea75ebd87eeb4d291 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:07.512371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:07.512828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:07.512920 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:07.530760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:07.531966 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:07.535847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=100 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:07.590672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1364/2748] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:07.600602 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:07.601991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:07.609230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:07.667152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57603aa2-66b0-4d84-ac60-a2a2b161a0b0 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:07.671436 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:07.672087 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:07.672369 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:07.672644 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:07.683108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:07.684583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab06b2-ff8e-45ee-89e3-a3438e1cb2b0 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:07.695439 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1385/2749] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:07.869437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:07.870728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:07.892001 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96715e5-1e04-4ee9-b6fb-6de9f2108105 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:07.897551 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:07.898478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:07.898760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:07.899060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] PATH_INFO: `/v3/projects/69923bd737bb4772852723c5cd99d57f` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:07.900674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69923bd737bb4772852723c5cd99d57f)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:07.902554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:07.958918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69923bd737bb4772852723c5cd99d57f', 'request_id': 'req-96788c8c-5cd9-4de9-84e0-630043d2cc45'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.965393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69923bd737bb4772852723c5cd99d57f', 'request_id': 'req-96788c8c-5cd9-4de9-84e0-630043d2cc45'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.971816 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69923bd737bb4772852723c5cd99d57f', 'request_id': 'req-96788c8c-5cd9-4de9-84e0-630043d2cc45'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.984432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] The token cache is being invalidate because project 69923bd737bb4772852723c5cd99d57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:07.984721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.985343 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-96788c8c-5cd9-4de9-84e0-630043d2cc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:07.986963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1365/2750] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:07 2022] DELETE /identity/v3/projects/69923bd737bb4772852723c5cd99d57f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:07.992188 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:07.992578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:07.992854 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.010420 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:08.011635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:08.071256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1386/2751] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:08.079577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f616-a488-4718-adbb-38463124875e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.079869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f616-a488-4718-adbb-38463124875e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.080151 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290f616-a488-4718-adbb-38463124875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.098822 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-5290f616-a488-4718-adbb-38463124875e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:08.100047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-5290f616-a488-4718-adbb-38463124875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:08.108033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1366/2752] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:08.112278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.121924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e9d5-3ff8-4b29-a4cc-2b52ac89e218 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.127047 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.127694 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.127978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.128260 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] PATH_INFO: `/v3/domains` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.128970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91851) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:28:08.129275 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.130894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.143018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1387/2753] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:28:08.147278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290f616-a488-4718-adbb-38463124875e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.156716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290f616-a488-4718-adbb-38463124875e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.161405 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.162064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.162374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.162659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.163352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.165253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.191691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1367/2754] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:28:08 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:28:08.195625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.207658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa363-4059-44f1-afd3-ecef4c51e731 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.265218 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.265893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.266176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.266489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] PATH_INFO: `/v3/users` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.267216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.269134 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.290491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1388/2755] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:28:08 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:28:08.294346 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.305453 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f536b9-9d99-4036-a7fe-9ed9c2fc12f2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.310062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.310791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.311076 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.311350 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.312102 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.313698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.324413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1368/2756] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:08.328675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.338731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7b7-a6d7-4676-841d-010f7d308313 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.343963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.344623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.344910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.345185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.345904 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.347536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bacf20-f179-4dc9-ab6d-afbb4de3d3cd admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.358299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1389/2757] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:28:08.362510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.372581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23b72-02a5-4825-9b6f-2151e97350c9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.377130 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.377996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.378276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.378583 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] PATH_INFO: `/v3/projects/ad1262fe3341444aaa45621abeb94ef3/users/6945170d21604d8d81d623ed25a3925b/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.392996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1262fe3341444aaa45621abeb94ef3, user_id=6945170d21604d8d81d623ed25a3925b, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.395705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.409657 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1369/2758] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:28:08 2022] PUT /identity/v3/projects/ad1262fe3341444aaa45621abeb94ef3/users/6945170d21604d8d81d623ed25a3925b/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:08.421093 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.421383 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.421658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.446998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:08.450499 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] MFA Rules not processed for user `6945170d21604d8d81d623ed25a3925b`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:08.452208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:08.517603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1390/2759] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:28:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:28:08.528781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:08.531135 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:08.567224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:08.610194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72c2c9-b5e9-4131-b58f-15cbc8f6ab13 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:08.613901 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:08.614517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:08.614804 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:08.615088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:08.623309 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:08.624736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a9a6e-86a8-4012-9310-e56fa1f4adc6 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:08.633499 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1370/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:17.770738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:17.837988 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c190145-bb9c-4931-99da-a40f0a272ac4 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:17.843311 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:17.844199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:17.844514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:17.844788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] PATH_INFO: `/v3/users/6945170d21604d8d81d623ed25a3925b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:17.846029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=6945170d21604d8d81d623ed25a3925b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:17.848044 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:17.902830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6945170d21604d8d81d623ed25a3925b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:17.911023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6945170d21604d8d81d623ed25a3925b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:17.911655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6945170d21604d8d81d623ed25a3925b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:17.925068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6945170d21604d8d81d623ed25a3925b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:17.926079 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6945170d21604d8d81d623ed25a3925b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:17.942734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1391/2761] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:28:17 2022] DELETE /identity/v3/users/6945170d21604d8d81d623ed25a3925b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:17.948360 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:17.948946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:17.948946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:17.970993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:28:17.972348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:18.044841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1371/2762] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:28:18.055812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:18.057495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:18.087886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:18.153385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7001e1-11ce-4d0f-8a68-d3c7952f6dac admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:18.157265 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:18.157914 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:18.158197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:18.158504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:18.168558 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:18.170024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e603-22e3-4555-a168-5e52d7791c0f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:18.180326 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1392/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:28:18.506069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:28:18.508371 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:28:18.525218 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62771042-d6a2-480a-a029-e8ca4aec6359 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:28:18.530364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:28:18.531253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:28:18.531542 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:28:18.531815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] PATH_INFO: `/v3/projects/ad1262fe3341444aaa45621abeb94ef3` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:28:18.533486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1262fe3341444aaa45621abeb94ef3)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:28:18.535709 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:28:18.590883 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1262fe3341444aaa45621abeb94ef3', 'request_id': 'req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:18.597155 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1262fe3341444aaa45621abeb94ef3', 'request_id': 'req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:18.602729 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1262fe3341444aaa45621abeb94ef3', 'request_id': 'req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:18.615864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] The token cache is being invalidate because project ad1262fe3341444aaa45621abeb94ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:28:18.616148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:18.616773 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ec8924f0-42a2-48a6-a53c-5b9cfc8968ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:28:18.618382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1372/2764] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:28:18 2022] DELETE /identity/v3/projects/ad1262fe3341444aaa45621abeb94ef3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:28:28.175902 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=25 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:28:28.612733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:29:59.011809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.012435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.012616 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] PATH_INFO: `/` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.014127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1393/2765] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 18:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:29:59.020149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36b28-9377-449e-a557-77dd543e9075 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.020737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36b28-9377-449e-a557-77dd543e9075 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.020737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36b28-9377-449e-a557-77dd543e9075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.081259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-56e36b28-9377-449e-a557-77dd543e9075 None None] MFA Rules not processed for user `30dea44d32f0405a9344f004640d44a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:59.082526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-56e36b28-9377-449e-a557-77dd543e9075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:59.144457 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1373/2766] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:59.159565 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:59.162137 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:59.176965 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5488b-7f4b-4592-81cc-9876b258a532 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:59.180691 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:59.181342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.181628 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.181912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.189509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:59.190991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf736472-c80d-427e-82dd-6ad647dbac5d service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:59.200105 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1394/2767] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:29:59.289469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd3ae0-5473-46db-8572-96341da79e6b None None] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.289761 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd3ae0-5473-46db-8572-96341da79e6b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.290043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd3ae0-5473-46db-8572-96341da79e6b None None] PATH_INFO: `/` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.291504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1374/2768] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jun 30 18:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 18:29:59.297185 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.297495 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.297801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.348680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] MFA Rules not processed for user `c064d58c15704009864fc43a1e630676`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:29:59.349830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:29:59.472422 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1395/2769] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jun 30 18:29:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:29:59.489714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd3ae0-5473-46db-8572-96341da79e6b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:29:59.507715 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd3ae0-5473-46db-8572-96341da79e6b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:29:59.513848 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:29:59.514622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:29:59.515050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:29:59.515354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:29:59.530125 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:29:59.533365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:29:59.556301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1375/2770] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:09.466509 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=55 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:09.545320 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=39 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:12.032025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:12.033578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:12.048581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2881c6-917a-49bb-ac4d-f206e14df05e None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:12.054767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:12.055480 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:12.055829 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:12.056148 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:12.058173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:12.120999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:12.122634 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:12.133167 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1396/2771] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:14.512568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:14.522095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb16d-38d8-4b07-80a4-be4a28f8ad0f service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:14.526432 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:14.527068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:14.527347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:14.527618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:14.529412 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:14.623292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:14.624737 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:14.648312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1376/2772] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:17.861692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:17.864071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:17.934126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:17.936561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:17.959807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7cfb6-699b-4f05-83f4-00e2788ccfd6 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:17.969167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:17.970659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:17.971183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:17.971681 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] PATH_INFO: `/v3/users/2ed5e98a29c44296aeb921af4f521042` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:17.973743 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed5e98a29c44296aeb921af4f521042)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:17.977625 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:17.990957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cd405-c3fe-4f9f-9f70-725595af6c00 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:17.996909 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:17.997643 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:17.998056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:17.998459 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.000343 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:18.013950 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.015729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.031307 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1397/2773] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:30:18.036667 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed5e98a29c44296aeb921af4f521042'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.037336 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed5e98a29c44296aeb921af4f521042'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.054445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed5e98a29c44296aeb921af4f521042'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.064248 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed5e98a29c44296aeb921af4f521042'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.065025 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-bc0de9be-ea4c-43a1-8f04-af5934da3add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed5e98a29c44296aeb921af4f521042'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.082527 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1377/2774] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:17 2022] DELETE /identity/v3/users/2ed5e98a29c44296aeb921af4f521042 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:18.267833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.282059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1c8cd-19df-4da3-b743-156741d1c05b service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.346578 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.347441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.347721 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.347992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] PATH_INFO: `/v3/projects/eacf1ab0e2244313b8f4f04d5d5202fa` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.349207 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacf1ab0e2244313b8f4f04d5d5202fa)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.351549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.394702 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:18.409604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacf1ab0e2244313b8f4f04d5d5202fa', 'request_id': 'req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.415504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacf1ab0e2244313b8f4f04d5d5202fa', 'request_id': 'req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.426245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacf1ab0e2244313b8f4f04d5d5202fa', 'request_id': 'req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.439467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] The token cache is being invalidate because project eacf1ab0e2244313b8f4f04d5d5202fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:18.439754 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.440419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-8aca9a58-a0cc-4412-91ca-c81e3d3ec15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:18.441986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1398/2775] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:18 2022] DELETE /identity/v3/projects/eacf1ab0e2244313b8f4f04d5d5202fa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:18.448944 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.449229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.449754 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.498921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:18.500138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:18.510242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:18.582927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1378/2776] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:18.597894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.598180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.598493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.618639 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:18.619895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:18.628229 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1399/2777] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:18.633440 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.646111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d38f0-4225-427b-9c6e-1efadd799786 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.651112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.651778 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.652157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.652330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.653028 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:18.653334 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.654985 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.672565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1379/2778] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:30:18.677402 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.688147 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ec6ea-f2e5-4c7f-8c8d-4a2869cdf350 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.699613 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.700927 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.701339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.701622 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.702539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.704409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.760647 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1400/2779] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:18 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:18.761963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.773429 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4694f-125a-4199-bdaf-4ed272dfc551 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.868100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.868787 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.869075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.869354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.870062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.872106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.895861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1380/2780] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:18 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:18.900064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.911917 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862e665-15c2-460d-a95f-dbcff47b7abe admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.918037 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.918776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.919063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.919336 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.923373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.923373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.936021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1401/2781] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:18.940082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.950452 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d8ef8-602b-4545-89d3-07c6b1421eab admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.955331 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.955935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.956243 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.956520 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:18.957284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:18.958930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018371e-3524-40df-96e2-2f07469eff7a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:18.970606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1381/2782] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:18.977277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:18.989449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecea8c3-8b1a-4b34-877c-b1bae0e090d4 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:18.994903 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:18.995783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:18.996063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:18.996355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] PATH_INFO: `/v3/projects/0e710eea71074319b0d07f2563a7a731/users/fa230b2592914be39f31f8a66bf4c648/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.012572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e710eea71074319b0d07f2563a7a731, user_id=fa230b2592914be39f31f8a66bf4c648, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.015346 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.030012 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1402/2783] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:18 2022] PUT /identity/v3/projects/0e710eea71074319b0d07f2563a7a731/users/fa230b2592914be39f31f8a66bf4c648/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:19.040954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.041244 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.041528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.068949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] MFA Rules not processed for user `fa230b2592914be39f31f8a66bf4c648`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:19.070269 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:19.134893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1382/2784] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:19.139340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.214494 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bce2b-3782-4551-a259-8921ad18a67b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.219273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.219938 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.220220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.220493 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.221208 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.223158 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.251823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1403/2785] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:19 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:19.256378 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.267727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e0716-2a02-4a90-a565-bd89d2122b6a None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.276150 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:19.329349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.330021 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.330300 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.330600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.331310 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.333217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.356448 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1383/2786] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:19 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:19.357693 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.368921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e353b3-3067-43c2-94ae-f2f440dcf04c admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.374050 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.374747 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.375034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.375314 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.376032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.377642 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.389293 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1404/2787] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:19.393525 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.404348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c983810-6eef-45b4-ae8c-454ec27a40e2 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.409113 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.409994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.410278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.410587 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] PATH_INFO: `/v3/projects/dd7156660e8e4b2cb7d278810cba651b/users/cc4eaaca50984f6eb913b38ce7b230f9/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.427146 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7156660e8e4b2cb7d278810cba651b, user_id=cc4eaaca50984f6eb913b38ce7b230f9, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.429921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.443818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1384/2788] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:19 2022] PUT /identity/v3/projects/dd7156660e8e4b2cb7d278810cba651b/users/cc4eaaca50984f6eb913b38ce7b230f9/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:19.448305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.459116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef5eb-c345-4197-9cb1-e99bf35ccab2 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.476344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:19.520763 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.521432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.521714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.521991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.523351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.524934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0da1e-5a89-43e2-8c3c-0ee25374238f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.535252 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1405/2789] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:19.539409 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.550075 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e0208-6f1e-45bc-a076-3054e8a9dc04 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.554886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.555840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.556121 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.556392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] PATH_INFO: `/v3/projects/dd7156660e8e4b2cb7d278810cba651b/users/cc4eaaca50984f6eb913b38ce7b230f9/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.558475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7156660e8e4b2cb7d278810cba651b, user_id=cc4eaaca50984f6eb913b38ce7b230f9, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.561115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.572698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1385/2790] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:19 2022] PUT /identity/v3/projects/dd7156660e8e4b2cb7d278810cba651b/users/cc4eaaca50984f6eb913b38ce7b230f9/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:19.585481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.585774 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.586056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.615025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] MFA Rules not processed for user `cc4eaaca50984f6eb913b38ce7b230f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:19.616262 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:19.690649 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1406/2791] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:19.703286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.705280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:19.775579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9cc77-e06f-401a-9f52-705d88a72762 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.779370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.780022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.780304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.780576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.790132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.791624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.802352 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1386/2792] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:19.880235 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:19.891100 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e15b21-67ae-4799-bb67-4e97d6258d8d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:19.894736 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:19.895418 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:19.895781 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:19.896119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:19.962827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:19.964301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac967ea-cd90-45cf-b01a-d2a10936607f service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:19.974718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1407/2793] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:29.797355 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:29.969883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=82 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:31.953536 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.018167 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f49c20-c2b6-43b5-ad6e-95a9d9d0225d service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.023164 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.024017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.024291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.024556 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] PATH_INFO: `/v3/users/fa230b2592914be39f31f8a66bf4c648` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.025733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa230b2592914be39f31f8a66bf4c648)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.027674 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.074012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa230b2592914be39f31f8a66bf4c648'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.074679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa230b2592914be39f31f8a66bf4c648'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.086735 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa230b2592914be39f31f8a66bf4c648'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.094317 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa230b2592914be39f31f8a66bf4c648'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.095017 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa230b2592914be39f31f8a66bf4c648'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.108103 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1387/2794] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:31 2022] DELETE /identity/v3/users/fa230b2592914be39f31f8a66bf4c648 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:32.112585 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.113085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.113181 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.129805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:32.131567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.191636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1408/2795] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:32.201780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.203229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.261307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f66c87-0503-46d7-9e9c-d0de79c24ff5 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.264946 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.265600 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.265879 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.266174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.276565 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.277989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.288046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1388/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:32.576251 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.577766 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:32.590811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f834d9-9ffd-4be9-8b97-cfa52675caab None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.597466 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.598435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.598725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.599072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] PATH_INFO: `/v3/projects/0e710eea71074319b0d07f2563a7a731` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.604392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e710eea71074319b0d07f2563a7a731)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.604392 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.658982 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e710eea71074319b0d07f2563a7a731', 'request_id': 'req-77865245-2bb9-4d88-ba07-a66134a2f682'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.664794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e710eea71074319b0d07f2563a7a731', 'request_id': 'req-77865245-2bb9-4d88-ba07-a66134a2f682'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.672154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e710eea71074319b0d07f2563a7a731', 'request_id': 'req-77865245-2bb9-4d88-ba07-a66134a2f682'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.684783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] The token cache is being invalidate because project 0e710eea71074319b0d07f2563a7a731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:32.685075 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.685735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.687409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1409/2797] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:32 2022] DELETE /identity/v3/projects/0e710eea71074319b0d07f2563a7a731 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:32.691480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:32.702008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9356c8-d9d2-4af7-b7e1-5b02472a32d3 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:32.721134 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:32.766293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:32.767207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:32.767484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:32.767757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] PATH_INFO: `/v3/users/cc4eaaca50984f6eb913b38ce7b230f9` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:32.768971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4eaaca50984f6eb913b38ce7b230f9)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:32.770953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:32.820394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4eaaca50984f6eb913b38ce7b230f9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.821077 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4eaaca50984f6eb913b38ce7b230f9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.833321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4eaaca50984f6eb913b38ce7b230f9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.842307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4eaaca50984f6eb913b38ce7b230f9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.843088 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-917ab00c-4470-44ba-abee-7e86d6e42ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4eaaca50984f6eb913b38ce7b230f9'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:32.856062 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1389/2798] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:32 2022] DELETE /identity/v3/users/cc4eaaca50984f6eb913b38ce7b230f9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:33.158714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.160177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.174887 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865245-2bb9-4d88-ba07-a66134a2f682 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.247581 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.248457 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.248788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.249106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] PATH_INFO: `/v3/projects/dd7156660e8e4b2cb7d278810cba651b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.250554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7156660e8e4b2cb7d278810cba651b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.252684 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.287954 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:33.306403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7156660e8e4b2cb7d278810cba651b', 'request_id': 'req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:33.312538 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7156660e8e4b2cb7d278810cba651b', 'request_id': 'req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:33.320864 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7156660e8e4b2cb7d278810cba651b', 'request_id': 'req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:33.335023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] The token cache is being invalidate because project dd7156660e8e4b2cb7d278810cba651b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:33.335313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:33.336028 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a5d6bf15-e36e-4a98-ae53-29b3074f3b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:33.337587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1410/2799] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:33 2022] DELETE /identity/v3/projects/dd7156660e8e4b2cb7d278810cba651b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:33.342668 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.343183 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.343364 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.362052 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:33.363308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.429969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1390/2800] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:30:33.438614 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.438956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.439239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.464869 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:33.466139 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.474258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1411/2801] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:33.479295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.489393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041d361-55a1-473b-af3f-7a83720f486e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.494526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.495201 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.495483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.495757 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.496517 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:33.496858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.498611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.511210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1391/2802] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 18:30:33.515852 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.526125 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b951a-9a28-4f8f-99da-35ba3c9ece0d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.530979 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.531633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.531913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.532187 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.532876 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.534872 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.564776 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1412/2803] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:33.569845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.580796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e986eb9-2d05-48e6-9996-f9c5fa66fcbc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.609581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:33.647930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.648617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.648904 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.649185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.649891 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.651896 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.676369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1392/2804] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/users => generated 304 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:33.680928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.692183 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7ea20-797d-4d86-ba3c-bb2fcf393463 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.697760 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.698461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.698738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.699009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.699740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.701371 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.713379 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1413/2805] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:30:33.717500 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.728041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336165cf-4e68-4093-8b15-86df13e36508 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.732900 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.733578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.733858 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.734138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.734927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.736547 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63ac7a-0bbb-40f2-a533-69c448ac05cc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.747387 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1393/2806] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:33.752180 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:33.763073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5611b3-8bf4-425d-a2d6-8b19426fc12e admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:33.767783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:33.768665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.768942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.769216 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] PATH_INFO: `/v3/projects/24240b66cd664dc282078ec4df7328f8/users/51431f22f10848398357ec19c5c985d4/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.785401 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24240b66cd664dc282078ec4df7328f8, user_id=51431f22f10848398357ec19c5c985d4, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:33.788246 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:33.806104 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1414/2807] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:33 2022] PUT /identity/v3/projects/24240b66cd664dc282078ec4df7328f8/users/51431f22f10848398357ec19c5c985d4/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:33.817390 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:33.817740 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:33.818078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:33.852362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] MFA Rules not processed for user `51431f22f10848398357ec19c5c985d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:33.853647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:33.930579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1394/2808] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:33.935220 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.007017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d36ea-b432-4e66-8510-de0c93d6c4b6 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.012173 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.012881 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.013171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.013445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.014152 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.016162 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.045596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1415/2809] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:33 2022] POST /identity/v3/projects => generated 409 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:34.050158 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.061903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c44c-2616-460f-96a4-b8d7c0525e23 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.121987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.122841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.123246 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.123661 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.124443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.126529 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.148445 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1395/2810] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:34 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:34.152749 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.164301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247d642-41b9-447c-baef-a52f065e34d1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.171206 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.172891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.173313 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.173837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.174606 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.176219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.188106 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1416/2811] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:34.192286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.202557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687eb00-ab4f-4476-a3b9-0044a6b1cc1d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.208753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.210313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.210689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.210989 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] PATH_INFO: `/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b/users/b9e8a65ba09b40e6a79dbd38af0878ec/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.227663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a3f10e9b6f42ea8481a83f2814a94b, user_id=b9e8a65ba09b40e6a79dbd38af0878ec, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.230558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.245921 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1396/2812] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:34 2022] PUT /identity/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b/users/b9e8a65ba09b40e6a79dbd38af0878ec/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:34.250109 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.260705 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108d35e-6d9a-4336-aff7-923deaab45d6 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.282540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:34.321855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.322549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.322838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.323119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.323846 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.325470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a97ade-5432-4699-9235-d9ac30f90e87 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.336886 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1417/2813] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:34.344932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.351072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:34.357492 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512f96-3685-4a01-b7aa-ad6b5d59a0f9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.362851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.363741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.364178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.364491 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] PATH_INFO: `/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b/users/b9e8a65ba09b40e6a79dbd38af0878ec/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.366633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a3f10e9b6f42ea8481a83f2814a94b, user_id=b9e8a65ba09b40e6a79dbd38af0878ec, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.369078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.380033 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1397/2814] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:34 2022] PUT /identity/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b/users/b9e8a65ba09b40e6a79dbd38af0878ec/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:34.392210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.392504 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.392891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.420949 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] MFA Rules not processed for user `b9e8a65ba09b40e6a79dbd38af0878ec`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:34.422223 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.498707 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1418/2815] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:34 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:34.510818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.512639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:34.576969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd994a-5199-495b-ad50-5e1cfa68a5aa admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.580796 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.581439 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.581723 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.581996 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.593024 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.594312 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.605128 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1398/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:34.777342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:34.788669 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7988-6435-4aaf-bf04-fdad95c2115b None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:34.793151 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:34.794006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:34.794290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:34.794607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:34.857805 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:34.859464 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341865cf-bf97-4fc3-814b-a83e0ab98a17 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:34.873076 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1419/2817] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:44.600301 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:44.866527 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=80 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:45.346388 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:45.408254 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9625-1283-4bb5-9433-047e488fd320 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:45.413873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:45.414770 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:45.415050 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:45.415321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] PATH_INFO: `/v3/users/51431f22f10848398357ec19c5c985d4` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:45.416495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51431f22f10848398357ec19c5c985d4)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:45.418434 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:45.465952 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51431f22f10848398357ec19c5c985d4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:45.466561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51431f22f10848398357ec19c5c985d4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:45.478070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51431f22f10848398357ec19c5c985d4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:45.485567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51431f22f10848398357ec19c5c985d4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:45.486354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51431f22f10848398357ec19c5c985d4'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:45.499131 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1399/2818] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:45 2022] DELETE /identity/v3/users/51431f22f10848398357ec19c5c985d4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:45.503762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:45.504046 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:45.504327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:45.521631 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:45.522865 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:45.582080 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1420/2819] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:45.591696 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:45.593089 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:45.651392 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a611efc-f496-423f-90e5-013aa23562c4 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:45.655068 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:45.655728 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:45.655992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:45.656267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:45.666935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:45.668368 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:45.678617 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1400/2820] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:45.947697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:45.949481 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:45.967626 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc117b34-72c5-41e0-a3bb-c6b8d038ebdc None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:45.972621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:45.973680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:45.973969 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:45.974806 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] PATH_INFO: `/v3/projects/24240b66cd664dc282078ec4df7328f8` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:45.975960 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=24240b66cd664dc282078ec4df7328f8)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:45.977945 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:46.026552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24240b66cd664dc282078ec4df7328f8', 'request_id': 'req-60f3dc54-739a-4023-a4f9-2a534875d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.031812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24240b66cd664dc282078ec4df7328f8', 'request_id': 'req-60f3dc54-739a-4023-a4f9-2a534875d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.040295 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24240b66cd664dc282078ec4df7328f8', 'request_id': 'req-60f3dc54-739a-4023-a4f9-2a534875d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.051862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] The token cache is being invalidate because project 24240b66cd664dc282078ec4df7328f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:46.052230 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.052913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.054489 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1421/2821] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:45 2022] DELETE /identity/v3/projects/24240b66cd664dc282078ec4df7328f8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:46.058437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:46.068229 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eff10e-c253-4d5f-9dbe-943f7010a9f4 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:46.087315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:46.132528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:46.133399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.133678 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.133995 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] PATH_INFO: `/v3/users/b9e8a65ba09b40e6a79dbd38af0878ec` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.135209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9e8a65ba09b40e6a79dbd38af0878ec)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:46.137126 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:46.184660 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9e8a65ba09b40e6a79dbd38af0878ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.185404 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9e8a65ba09b40e6a79dbd38af0878ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.199374 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9e8a65ba09b40e6a79dbd38af0878ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.210137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9e8a65ba09b40e6a79dbd38af0878ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.210866 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-63ba22ab-a35f-486c-98dd-9f2f7e2d5449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9e8a65ba09b40e6a79dbd38af0878ec'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.223955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1401/2822] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:46 2022] DELETE /identity/v3/users/b9e8a65ba09b40e6a79dbd38af0878ec => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:46.514347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:46.515814 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:46.530209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3dc54-739a-4023-a4f9-2a534875d5fb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:46.590391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:46.591178 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.591461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.591737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] PATH_INFO: `/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.593127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a3f10e9b6f42ea8481a83f2814a94b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:46.595286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:46.627670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:46.647531 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a3f10e9b6f42ea8481a83f2814a94b', 'request_id': 'req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.657339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a3f10e9b6f42ea8481a83f2814a94b', 'request_id': 'req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.665474 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a3f10e9b6f42ea8481a83f2814a94b', 'request_id': 'req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.677347 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] The token cache is being invalidate because project c5a3f10e9b6f42ea8481a83f2814a94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:46.677636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.678284 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-dfb2edd6-9f39-40b4-ba0d-1ea1d1a31b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:46.679943 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1422/2823] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:46 2022] DELETE /identity/v3/projects/c5a3f10e9b6f42ea8481a83f2814a94b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:46.684790 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.685332 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.685426 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.703747 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:46.704962 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:46.767047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1402/2824] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:46.775400 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.775714 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.775992 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.794880 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:46.796120 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:46.804986 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1423/2825] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:46.809315 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:46.819646 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42983ffa-9d6e-448a-aca4-27ce93476f9b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:46.824190 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:46.824841 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.825120 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.825391 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.826109 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:46.826442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:46.828046 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:46.839639 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1403/2826] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:46.844289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:46.853584 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af7da3-a19d-4a87-967d-55b0f3982518 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:46.858487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:46.859149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.859433 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.859708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.860393 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:46.862543 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:46.893490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1424/2827] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:46 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:46.894644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:46.904774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab6316-b07f-461e-9847-6efccbaad907 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:46.942365 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:46.974470 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:46.975138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:46.975416 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:46.975685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:46.976370 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:46.978288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.003927 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1404/2828] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:46 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:47.005201 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.016344 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224e02a-1af4-4021-bc05-8a7aa1c7f315 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.021023 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.021681 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.021963 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.022240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.023009 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.024609 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.035259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1425/2829] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:47.039377 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.049736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4fd5d-c000-4455-aae0-4048e808b55a admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.055082 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.055760 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.056010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.056280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.057012 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.058618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96883-f219-4897-bcc1-1befba023ea7 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.068906 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1405/2830] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:47.073249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.083830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127eccf-510a-4cf3-af0b-353fd42ca1ac admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.088510 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.089375 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.089658 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.089933 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] PATH_INFO: `/v3/projects/5cde2a5c34074c5499881d125b854b8b/users/f237973399cf4ab38c969069933a52f7/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.105008 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cde2a5c34074c5499881d125b854b8b, user_id=f237973399cf4ab38c969069933a52f7, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.107651 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.121791 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1426/2831] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:47 2022] PUT /identity/v3/projects/5cde2a5c34074c5499881d125b854b8b/users/f237973399cf4ab38c969069933a52f7/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:47.132502 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.132791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.133073 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.158954 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] MFA Rules not processed for user `f237973399cf4ab38c969069933a52f7`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:47.160181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:47.225353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1406/2832] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:47.229855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.290154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1ae92-9d02-4ab0-ae18-5fe52ec1badf admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.294809 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.295452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.295733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.296005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.296718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.298897 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.324812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1427/2833] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:47 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:47.328504 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.339991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074aba3-34ac-498a-9223-059ad9d32d6e None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.396969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.397561 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.397828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.398097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.398818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.400732 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.420628 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1407/2834] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:47 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:30:47.425153 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.435799 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b19-3af3-4400-90a6-49653f2dddc5 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.440518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.441179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.441461 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.441733 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.442515 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.445713 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.456621 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1428/2835] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:47.461469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.471970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179807c3-e640-4dfb-8001-fef268aaecec admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.476701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.477577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.477861 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.478138 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] PATH_INFO: `/v3/projects/21015a1087544ceea83a85f35d7bd5e9/users/3bfa2d8ecc794a17825ad1c1ffacc314/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.494149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21015a1087544ceea83a85f35d7bd5e9, user_id=3bfa2d8ecc794a17825ad1c1ffacc314, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.496877 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.510676 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1408/2836] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:47 2022] PUT /identity/v3/projects/21015a1087544ceea83a85f35d7bd5e9/users/3bfa2d8ecc794a17825ad1c1ffacc314/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:47.514991 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.524901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683a91a-4ec0-4039-b586-18fdd4a15b15 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.543970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:47.582843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.583508 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.583795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.584073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.584793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.586446 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1edbf-ebf2-492b-8e41-9edbf1313205 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.596449 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1429/2837] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:47.600679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.606321 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:47.612654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf501ed-d103-4314-a301-8ce59d9531a9 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.618119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.619023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.619304 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.619578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] PATH_INFO: `/v3/projects/21015a1087544ceea83a85f35d7bd5e9/users/3bfa2d8ecc794a17825ad1c1ffacc314/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.621528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21015a1087544ceea83a85f35d7bd5e9, user_id=3bfa2d8ecc794a17825ad1c1ffacc314, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.624224 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.634977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1409/2838] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:47 2022] PUT /identity/v3/projects/21015a1087544ceea83a85f35d7bd5e9/users/3bfa2d8ecc794a17825ad1c1ffacc314/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:30:47.646060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.646373 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.646701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.673159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] MFA Rules not processed for user `3bfa2d8ecc794a17825ad1c1ffacc314`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:47.674289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:47.744686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1430/2839] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:47.756507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.758353 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:47.815916 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38731-dd9d-4f8b-8e2f-7f9a2d48a08b admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.820119 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.820756 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.821035 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.821308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.830539 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.831969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:47.841276 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1410/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:47.919156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:47.929925 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35251bd4-f6aa-44d2-b04f-5713f2d1ed83 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:47.933465 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:47.934103 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:47.934416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:47.934695 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:47.996712 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:47.998154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a388e1b-3a25-45c5-9f3e-10136a2801ca service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:48.007310 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1431/2841] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:52.728071 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:52.789992 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbbc2f-39f0-4c0a-a5db-c4206d911c11 service cinder] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:52.794701 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:52.795573 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:52.795859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:52.796139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] PATH_INFO: `/v3/users/f237973399cf4ab38c969069933a52f7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:52.797602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f237973399cf4ab38c969069933a52f7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:52.799710 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:52.847417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f237973399cf4ab38c969069933a52f7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:52.848005 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f237973399cf4ab38c969069933a52f7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:52.859994 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f237973399cf4ab38c969069933a52f7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:52.868533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f237973399cf4ab38c969069933a52f7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:52.869176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f237973399cf4ab38c969069933a52f7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:52.882171 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1411/2842] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:52 2022] DELETE /identity/v3/users/f237973399cf4ab38c969069933a52f7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:52.886706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:52.887182 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:52.887289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:52.905035 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:52.906270 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:52.948500 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:52.968491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1432/2843] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:52.978069 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:52.979526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:52.994095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:53.039890 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704a4ce-3e68-4b3d-a3d2-e07625979432 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:53.043647 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:53.044289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:53.044581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:53.044859 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:53.055094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:53.056531 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:53.066010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1412/2844] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:30:53.329398 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:53.330831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:53.342710 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a1a65-2526-4b2e-b6ab-05f4b2d2a4ed None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:53.347664 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:53.348530 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:53.349320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:53.349592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] PATH_INFO: `/v3/projects/5cde2a5c34074c5499881d125b854b8b` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:53.350813 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cde2a5c34074c5499881d125b854b8b)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:53.352737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:53.400800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cde2a5c34074c5499881d125b854b8b', 'request_id': 'req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.406217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cde2a5c34074c5499881d125b854b8b', 'request_id': 'req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.414732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cde2a5c34074c5499881d125b854b8b', 'request_id': 'req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.426516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] The token cache is being invalidate because project 5cde2a5c34074c5499881d125b854b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:53.426855 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.427994 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.429570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1433/2845] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:53 2022] DELETE /identity/v3/projects/5cde2a5c34074c5499881d125b854b8b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:53.433629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:53.443285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b41d31-461a-447d-a23c-4f92e0461bdf service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:53.498744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:53.499611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:53.499893 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:53.500172 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] PATH_INFO: `/v3/users/3bfa2d8ecc794a17825ad1c1ffacc314` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:53.501363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfa2d8ecc794a17825ad1c1ffacc314)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:53.503314 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:53.549679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfa2d8ecc794a17825ad1c1ffacc314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.550363 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfa2d8ecc794a17825ad1c1ffacc314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.562885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfa2d8ecc794a17825ad1c1ffacc314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.571460 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfa2d8ecc794a17825ad1c1ffacc314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.572115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-4c3cf36e-c35d-4d74-9071-371827c51aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfa2d8ecc794a17825ad1c1ffacc314'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:53.585864 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1413/2846] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:30:53 2022] DELETE /identity/v3/users/3bfa2d8ecc794a17825ad1c1ffacc314 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:53.869228 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:53.870738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:53.885844 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe66a-fea7-4fd6-9b5a-ce1436f9c697 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:53.958034 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:53.959517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:53.959722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:53.959957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] PATH_INFO: `/v3/projects/21015a1087544ceea83a85f35d7bd5e9` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:53.961168 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21015a1087544ceea83a85f35d7bd5e9)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:53.963665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.017830 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21015a1087544ceea83a85f35d7bd5e9', 'request_id': 'req-a4614ad2-5901-4176-a35b-678297a7d3a3'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:54.024479 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21015a1087544ceea83a85f35d7bd5e9', 'request_id': 'req-a4614ad2-5901-4176-a35b-678297a7d3a3'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:54.032219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21015a1087544ceea83a85f35d7bd5e9', 'request_id': 'req-a4614ad2-5901-4176-a35b-678297a7d3a3'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:54.039435 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:54.045083 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] The token cache is being invalidate because project 21015a1087544ceea83a85f35d7bd5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:30:54.045607 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:54.046729 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a4614ad2-5901-4176-a35b-678297a7d3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:30:54.048908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1434/2847] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:30:53 2022] DELETE /identity/v3/projects/21015a1087544ceea83a85f35d7bd5e9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:54.053905 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.054413 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.054510 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.074486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:54.075755 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:54.105078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:30:54.137515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1414/2848] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:54.145247 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.145534 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.146388 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.164550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:54.165773 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:54.174029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1435/2849] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:54.178291 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.187975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ffa42-0442-4822-bc39-2e50787fdc37 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.192963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.193611 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.193888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.194162 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.194976 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:30:54.195249 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.196847 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.209139 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1415/2850] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:30:54.213635 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.222932 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfc5d4-acfc-4955-b868-7045e5d14d33 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.227860 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.228526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.228811 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.229116 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.229810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.231757 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.259537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1436/2851] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:30:54 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:54.263868 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.273865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faf17e-9c2e-489a-9eda-54ce07120073 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.329811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.330503 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.330786 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.331058 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.331744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.333680 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.352971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1416/2852] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:30:54 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:30:54.357063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.367944 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f494348-576c-4c7e-a851-6adaa68282a7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.373081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.373740 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.374021 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.374291 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.375038 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.376633 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.388163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1437/2853] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:54.392267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.406324 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9ba8f-383e-4b0a-9045-d3bf56855569 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.410951 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.411739 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.411926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.412204 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.412963 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.414592 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756dfa5-e747-4360-9545-af582ada8437 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.425124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1417/2854] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:30:54.429753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.439665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774c29-e74a-4f02-a5ee-f09d0b32837f admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.444923 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.445788 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.446068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.446452 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] PATH_INFO: `/v3/projects/d784e6efdb55451199f72fdbf52b2220/users/c3cb59f1232345d69ca5514f1cd8a794/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.460724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d784e6efdb55451199f72fdbf52b2220, user_id=c3cb59f1232345d69ca5514f1cd8a794, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.463340 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.476467 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1438/2855] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:30:54 2022] PUT /identity/v3/projects/d784e6efdb55451199f72fdbf52b2220/users/c3cb59f1232345d69ca5514f1cd8a794/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:30:54.487643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651e16-6838-4268-985c-339220e18369 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.487933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651e16-6838-4268-985c-339220e18369 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.488209 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651e16-6838-4268-985c-339220e18369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.516278 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-38651e16-6838-4268-985c-339220e18369 None None] MFA Rules not processed for user `c3cb59f1232345d69ca5514f1cd8a794`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:30:54.517508 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-38651e16-6838-4268-985c-339220e18369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:54.582981 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1418/2856] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:30:54.593758 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:30:54.595488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:30:54.653194 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e0131-3f35-43f4-9696-bbf2095ebd81 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:30:54.656940 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:30:54.657591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:30:54.657874 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:30:54.658149 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:30:54.667470 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:30:54.668906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004f004-ead1-4687-bf98-00c1b1009b08 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:30:54.678738 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1439/2857] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:30:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:04.579928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:04.673999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:06.186221 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651e16-6838-4268-985c-339220e18369 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:06.248636 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651e16-6838-4268-985c-339220e18369 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:06.253080 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:06.253953 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:06.254251 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:06.255112 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] PATH_INFO: `/v3/users/c3cb59f1232345d69ca5514f1cd8a794` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:06.256297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cb59f1232345d69ca5514f1cd8a794)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:06.258207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:06.305835 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cb59f1232345d69ca5514f1cd8a794'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.306522 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cb59f1232345d69ca5514f1cd8a794'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.319515 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cb59f1232345d69ca5514f1cd8a794'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.326969 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cb59f1232345d69ca5514f1cd8a794'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.327603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cb59f1232345d69ca5514f1cd8a794'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.339898 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1419/2858] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:06 2022] DELETE /identity/v3/users/c3cb59f1232345d69ca5514f1cd8a794 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:06.344473 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:06.344948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:06.345069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:06.361583 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:06.362827 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:06.429919 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1440/2859] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:06.439653 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:06.441067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:06.497563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d0916-d12c-4a3e-aed4-e60243ab4b67 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:06.501295 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:06.501874 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:06.502454 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:06.502748 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:06.513828 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:06.515305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e9327-912f-4a2b-b2f4-c9881996e7c1 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:06.525781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1420/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:06.806952 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:06.808540 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:06.821385 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51bbb-3bd1-4215-b7c3-93f1fabeac57 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:06.826366 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:06.827349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:06.827737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:06.828017 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] PATH_INFO: `/v3/projects/d784e6efdb55451199f72fdbf52b2220` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:06.829213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d784e6efdb55451199f72fdbf52b2220)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:06.831224 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:06.885319 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd784e6efdb55451199f72fdbf52b2220', 'request_id': 'req-087841f9-f890-44e3-a3bd-6e570099cc9c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.891163 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd784e6efdb55451199f72fdbf52b2220', 'request_id': 'req-087841f9-f890-44e3-a3bd-6e570099cc9c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.900942 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd784e6efdb55451199f72fdbf52b2220', 'request_id': 'req-087841f9-f890-44e3-a3bd-6e570099cc9c'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.913328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] The token cache is being invalidate because project d784e6efdb55451199f72fdbf52b2220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:06.913663 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.914427 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-087841f9-f890-44e3-a3bd-6e570099cc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:06.916108 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1441/2861] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:06 2022] DELETE /identity/v3/projects/d784e6efdb55451199f72fdbf52b2220 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:06.921132 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:06.921422 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:06.922240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:06.940544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:06.942263 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:06.960484 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:07.006733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1421/2862] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:07.014176 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.014497 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.014779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.036912 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:07.038127 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.046056 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1442/2863] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:07.050679 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.063207 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5df434-e563-4579-b54f-108aeff415c2 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.069090 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.069808 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.070219 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.070544 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.071349 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:07.071658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.073319 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.087267 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1422/2864] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:07.092085 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.102164 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62bb6-24bf-46c1-ae86-d1ec87548b55 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.106928 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.107587 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.107998 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.108278 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.108977 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.111195 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.139990 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1443/2865] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:07.144537 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.154744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac518b-514c-4303-a8b5-682b9919bd6d admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.211929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.212689 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.212873 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.213149 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.213865 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.215832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.236163 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1423/2866] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:07.240934 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.251517 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00c64e-f3a1-432a-9b37-b134c6884825 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.256946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.257550 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.257837 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.258113 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.258878 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.260526 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.271441 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1444/2867] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:07.275468 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.285475 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9eb66-1cca-4af9-a252-3aefe5466ad3 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.290070 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.290764 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.291049 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.291325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.292045 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.293658 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b94dc2-1adb-43e9-bacf-c289410c54c4 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.305047 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1424/2868] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:07.309342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.320807 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb7a5c-56cc-4468-9197-e0fd7689d736 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.325838 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.326794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.327081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.327360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] PATH_INFO: `/v3/projects/26d8252ce44d4139accf6c9723c22bb1/users/e892bad3ac3e4044b0893b50aaf875c7/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.343948 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d8252ce44d4139accf6c9723c22bb1, user_id=e892bad3ac3e4044b0893b50aaf875c7, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.346682 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.361245 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1445/2869] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:07 2022] PUT /identity/v3/projects/26d8252ce44d4139accf6c9723c22bb1/users/e892bad3ac3e4044b0893b50aaf875c7/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:07.373644 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.374145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.374925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.414280 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] MFA Rules not processed for user `e892bad3ac3e4044b0893b50aaf875c7`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:07.414975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.493545 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1425/2870] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:07.526893 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.529002 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.552802 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:07.591304 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf48a3-e43c-4aff-a652-8ec84e21bf36 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.594970 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.595611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.595894 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.596169 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.605292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.606716 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d9ac-833d-4cf3-b48b-c92575602f6a service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.616156 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1446/2871] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:07.630107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.649029 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:07.697406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2bb39-deef-4768-b03f-8d5763c3111b None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.702016 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.702915 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.703194 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.703489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] PATH_INFO: `/v3/users/e892bad3ac3e4044b0893b50aaf875c7` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.704697 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e892bad3ac3e4044b0893b50aaf875c7)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.706685 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.753212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e892bad3ac3e4044b0893b50aaf875c7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.753923 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e892bad3ac3e4044b0893b50aaf875c7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.765721 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e892bad3ac3e4044b0893b50aaf875c7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.775881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e892bad3ac3e4044b0893b50aaf875c7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.776516 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e892bad3ac3e4044b0893b50aaf875c7'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:07.793297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1426/2872] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:31:07 2022] DELETE /identity/v3/users/e892bad3ac3e4044b0893b50aaf875c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:07.798209 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.798539 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.798816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.816058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:07.817283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.878315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1447/2873] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:07.888018 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:07.889480 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:07.947297 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87f5f9-ac76-4230-956f-243d3ef1cefa admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:07.950930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:07.951576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:07.951860 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:07.952136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:07.962274 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:07.963780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c05ec-1d9b-4127-ba03-73d74933e98a service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:07.974225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1427/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:08.241824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.243386 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.257957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca3486-0561-412b-be64-47c16dcb3a98 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.262680 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.263745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.264032 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.264308 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] PATH_INFO: `/v3/projects/26d8252ce44d4139accf6c9723c22bb1` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.265793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d8252ce44d4139accf6c9723c22bb1)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.267665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.315592 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d8252ce44d4139accf6c9723c22bb1', 'request_id': 'req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.321536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d8252ce44d4139accf6c9723c22bb1', 'request_id': 'req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.328843 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d8252ce44d4139accf6c9723c22bb1', 'request_id': 'req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.340551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] The token cache is being invalidate because project 26d8252ce44d4139accf6c9723c22bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:31:08.340831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.341476 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-cacc86d5-b534-4759-8b6a-a0a82fe5d5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:31:08.343033 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1448/2875] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:31:08 2022] DELETE /identity/v3/projects/26d8252ce44d4139accf6c9723c22bb1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:08.347807 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.348406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.348406 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.366800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.368008 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.401469 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:08.428188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1428/2876] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.436706 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.437256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.437256 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.456417 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.458212 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.467005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1449/2877] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.471241 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.481115 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03b3e4-cfd4-477e-90bc-5b83ecbc1980 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.485654 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.486307 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.486622 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.486899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.487619 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:31:08.487925 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.490022 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.501977 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1429/2878] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:31:08.507143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.517885 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73ef0a-8bd2-4cfd-a64c-122e4dd2ed9d None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.522800 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.523488 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.523744 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.524025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.524724 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.526793 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.554675 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1450/2879] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:08.559118 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.569854 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188aef7-dbfc-48d9-96ba-642682c77795 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.627403 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.628078 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.628380 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.628645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.629903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.631867 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.654145 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1430/2880] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:31:08.658572 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.669795 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b5ff-8a72-4d1f-8d08-e60ac995c4a1 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.674708 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.675372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.675650 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.675924 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.676672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.678259 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.689512 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1451/2881] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:08.693677 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.703698 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a85c9-a79a-4b39-8099-45f2c16b531f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.708791 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.709476 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.709781 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.710053 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.710800 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.712393 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41a69b-9df2-4670-a73e-4e496b943217 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.723441 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1431/2882] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:31:08.727918 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.737858 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525e19f-46e0-4e77-a9e9-7846a643e531 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.742456 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.743328 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.743603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.743875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] PATH_INFO: `/v3/projects/38d42cd833f64186913c2e46d0654994/users/a0b68c2c1bf8455abf675c38d1b7d7ee/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.746157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:08.762419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d42cd833f64186913c2e46d0654994, user_id=a0b68c2c1bf8455abf675c38d1b7d7ee, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.765305 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.778728 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1452/2883] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:31:08 2022] PUT /identity/v3/projects/38d42cd833f64186913c2e46d0654994/users/a0b68c2c1bf8455abf675c38d1b7d7ee/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:31:08.789518 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.789806 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.790081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.816099 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] MFA Rules not processed for user `a0b68c2c1bf8455abf675c38d1b7d7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:31:08.817540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.883302 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1432/2884] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:31:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:31:08.894429 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:31:08.896742 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:31:08.953416 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d752a2-e24b-4020-b83a-d063f2739b63 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:31:08.957197 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:31:08.958491 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:31:08.958779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:31:08.959058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:31:08.969025 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:31:08.970620 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce9224-9324-4e0e-b9b0-32fea7e06f99 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:31:08.980743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1453/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:31:08 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:31:18.880107 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:31:18.975898 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:00.743652 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:00.811605 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa6cfc-9cb2-43e0-8b17-eeea256e555f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:00.816072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:00.816937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:00.817217 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:00.817485 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] PATH_INFO: `/v3/users/a0b68c2c1bf8455abf675c38d1b7d7ee` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:00.818718 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b68c2c1bf8455abf675c38d1b7d7ee)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:00.820649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:00.866313 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b68c2c1bf8455abf675c38d1b7d7ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:00.867157 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b68c2c1bf8455abf675c38d1b7d7ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:00.878825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b68c2c1bf8455abf675c38d1b7d7ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:00.886292 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b68c2c1bf8455abf675c38d1b7d7ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:00.887359 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b68c2c1bf8455abf675c38d1b7d7ee'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:00.899673 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1433/2886] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:00 2022] DELETE /identity/v3/users/a0b68c2c1bf8455abf675c38d1b7d7ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:00.904339 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:00.905078 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:00.905258 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:00.925351 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:00.926617 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:00.987413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1454/2887] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:00.997707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:00.999212 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:01.057993 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988dd81-5160-406c-8efc-f9b62c9419e6 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.061612 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.062253 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.062572 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.062845 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.072602 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.074354 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee84fa-a036-4fdf-957c-852e1afbb056 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.085467 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1434/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:01.362277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.363818 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:01.377394 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c5b44-af74-45f5-bdfb-648921c7ca27 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.382179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.383070 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.383354 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.383630 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] PATH_INFO: `/v3/projects/38d42cd833f64186913c2e46d0654994` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.384824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d42cd833f64186913c2e46d0654994)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.386812 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.435816 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d42cd833f64186913c2e46d0654994', 'request_id': 'req-a683e655-0682-4700-9251-9599779c5e5d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:01.441753 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d42cd833f64186913c2e46d0654994', 'request_id': 'req-a683e655-0682-4700-9251-9599779c5e5d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:01.449273 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d42cd833f64186913c2e46d0654994', 'request_id': 'req-a683e655-0682-4700-9251-9599779c5e5d'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:01.461330 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] The token cache is being invalidate because project 38d42cd833f64186913c2e46d0654994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:01.461656 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:01.462286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-a683e655-0682-4700-9251-9599779c5e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:01.463847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1455/2889] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:01 2022] DELETE /identity/v3/projects/38d42cd833f64186913c2e46d0654994 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:01.468880 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.469206 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.469487 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.489040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:01.490399 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:01.508311 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:01.552550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1435/2890] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:01.561240 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.561519 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.561794 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.581005 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:01.582202 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:01.590192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1456/2891] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:01.594568 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.604421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ae720-d45b-48b5-9a03-c9fa4420d806 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.608967 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.609618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.609903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.610185 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] PATH_INFO: `/v3/domains` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.610983 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91852) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 18:32:01.611294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.612899 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.624633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1436/2892] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jun 30 18:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 18:32:01.629516 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.639029 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032e79d-d1e1-443b-89c2-1274314ec8a5 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.644737 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.645391 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.645672 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.645946 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.646722 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.648618 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.676596 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1457/2893] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:32:01 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:32:01.680818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.691424 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fab2e-e042-4f2f-bf7e-7315b95ab7bc admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.748079 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.748753 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.748950 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.749227 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.749922 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.752489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.773913 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1437/2894] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:32:01 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 18:32:01.778282 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.789655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853bcb1-1864-48ad-88ef-a083379bbf85 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.794431 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.795102 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.795387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.795926 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.796655 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.798301 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.814851 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1458/2895] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:32:01.818851 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.828948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec149a8-42de-43cf-80b6-f9731980ac34 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.833643 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.834284 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.834608 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.834886 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.835635 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.837231 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa8273-dd65-4c95-afe7-ee46829dbd40 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.847401 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1438/2896] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 18:32:01.851901 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:01.865553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a4fa-2318-4651-bc0b-c566ec02902d admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:01.870568 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:01.871451 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.871732 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.872004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] PATH_INFO: `/v3/projects/ad915a3e597f4730a63420d59470f5b6/users/40078727842f491bb566e205065a65ad/roles/0e71d3efba6d41a4a468c42031d0320e` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.894687 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad915a3e597f4730a63420d59470f5b6, user_id=40078727842f491bb566e205065a65ad, role_id=0e71d3efba6d41a4a468c42031d0320e)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:01.897521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:01.919111 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1459/2897] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:32:01 2022] PUT /identity/v3/projects/ad915a3e597f4730a63420d59470f5b6/users/40078727842f491bb566e205065a65ad/roles/0e71d3efba6d41a4a468c42031d0320e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:01.925187 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20de5-bb64-4012-a88c-5f4822715d75 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:01.925465 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20de5-bb64-4012-a88c-5f4822715d75 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:01.925749 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a20de5-bb64-4012-a88c-5f4822715d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:01.944330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-28a20de5-bb64-4012-a88c-5f4822715d75 None None] MFA Rules not processed for user `5c7b0226fd854776bbee2a2d0d0ee93a`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:01.945578 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-28a20de5-bb64-4012-a88c-5f4822715d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:02.007832 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1439/2898] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:02.017350 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:02.019066 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:02.042039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:02.079591 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d2e38-31dd-4d31-94bc-a3669f088b06 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:02.083204 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:02.084128 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:02.084413 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:02.084686 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:02.093549 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:02.095077 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:02.104636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1460/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:06.140649 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.141225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.141225 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.171659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] MFA Rules not processed for user `40078727842f491bb566e205065a65ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:06.172869 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:06.191554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:06.256576 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1440/2900] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:06.261000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.271524 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69b364-fab0-44d0-8b4e-de4145227dfd service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.282406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.286498 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.286784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.287063 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.287783 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.289780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.317955 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1461/2901] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jun 30 18:32:06 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:32:06.322043 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.333909 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7912f-727b-4061-a93b-1b58bef98a24 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.395970 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.396655 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.396938 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.397242 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] PATH_INFO: `/v3/users` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.397948 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.399928 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.427881 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1441/2902] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jun 30 18:32:06 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 18:32:06.432011 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.443073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf1d98-4b2e-48ba-9d0b-a386a1b6f1e7 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.448004 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.448772 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.449043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.449360 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.450041 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.451692 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.466058 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1462/2903] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:32:06.470929 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.481987 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f0b3e-8093-475d-9e01-63e868eb270f admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.487340 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.488252 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.488538 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.488818 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] PATH_INFO: `/v3/projects/98c62fb5effe42cf8234cb62c82e05fc/users/a0b85ec7b53a4230aff58e4b376ce163/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.508296 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c62fb5effe42cf8234cb62c82e05fc, user_id=a0b85ec7b53a4230aff58e4b376ce163, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.511040 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.526181 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1442/2904] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:32:06 2022] PUT /identity/v3/projects/98c62fb5effe42cf8234cb62c82e05fc/users/a0b85ec7b53a4230aff58e4b376ce163/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:06.530292 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.542064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061622e-f495-44f8-8db2-e962dc0a4c14 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.603365 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.604059 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.604342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.604624 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] PATH_INFO: `/v3/roles` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.605355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.607060 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a95eb-02c9-422f-b0f9-b74e178b8923 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.618623 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1463/2905] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jun 30 18:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 18:32:06.622693 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:06.635030 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acea39d-8d4b-41a9-bca3-b886ac421c0e admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:06.640095 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:06.641101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] REQUEST_METHOD: `PUT` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:06.641382 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:06.641656 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] PATH_INFO: `/v3/projects/98c62fb5effe42cf8234cb62c82e05fc/users/a0b85ec7b53a4230aff58e4b376ce163/roles/1df4096f023c48ccab33e5939498b3da` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:06.644285 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c62fb5effe42cf8234cb62c82e05fc, user_id=a0b85ec7b53a4230aff58e4b376ce163, role_id=1df4096f023c48ccab33e5939498b3da)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:06.647106 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:06.665010 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1443/2906] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jun 30 18:32:06 2022] PUT /identity/v3/projects/98c62fb5effe42cf8234cb62c82e05fc/users/a0b85ec7b53a4230aff58e4b376ce163/roles/1df4096f023c48ccab33e5939498b3da => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:11.104303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:11.104833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:11.104833 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:11.134190 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] MFA Rules not processed for user `a0b85ec7b53a4230aff58e4b376ce163`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:11.135384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:11.211746 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1464/2907] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:11.229935 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:11.232768 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:11.360318 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f2fa5-50c4-4618-9f2d-f6754d480695 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:11.369933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:11.371136 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:11.371659 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:11.372176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:11.382178 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:11.442888 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:11.444528 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:11.455477 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1444/2908] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:20.439409 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:20.453743 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfb281-2c4d-45d9-aebb-e926d63e4301 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:20.471646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:20.472432 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:20.472688 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:20.472875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:20.474831 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:20.489667 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:20.491322 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:20.498779 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:20.512975 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1465/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:20.640615 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:20.643097 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:20.710286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe37c-ad74-449d-a623-fc41684d78a0 service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:20.716272 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:20.717100 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:20.717435 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:20.717717 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:20.720348 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:20.736930 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:20.738769 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:20.751885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1445/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:21.877387 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:21.887321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551afdcd-711e-42ee-995e-d2077b7b4dfa service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:21.891159 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:21.891895 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:21.892177 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:21.892469 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:21.894399 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:21.904804 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:21.906239 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:21.915302 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1466/2911] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:21.989478 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:21.998631 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323063-7024-4a10-b472-a68749bd002c service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:22.002627 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:22.003306 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:22.003580 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:22.003885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:22.005688 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:22.020188 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:22.021626 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:22.034174 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1446/2912] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:23.170567 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:23.172312 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:23.270561 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b1099-8fe6-4aea-8297-7972e6fdd66f service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:23.278421 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:23.279444 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:23.279810 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:23.280143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:23.283619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:23.299342 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:23.301073 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:23.313018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1467/2913] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:32:24.119455 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:24.121596 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:24.145682 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0298f-814d-4d95-a063-2e68411d7f51 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:24.152852 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:24.153507 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:24.153805 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:24.154067 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:24.155957 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:24.170483 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:24.171579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:24.188372 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1447/2914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:25.834426 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:25.843929 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c95b1-7160-4e24-b746-0267be3ea2fe service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:25.848084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:25.848718 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:25.849003 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:25.849283 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:25.851057 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:25.914629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:25.916064 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:25.927822 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1468/2915] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:32.130691 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:32.132165 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:32.145750 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d160422-9b24-48ce-8a5d-08ae8947b2b7 service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:32.150514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:32.151383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:32.151663 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:32.151931 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] PATH_INFO: `/v3/users/82765721711e484d8bc87340177657cc` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:32.153116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82765721711e484d8bc87340177657cc)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:32.155104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:32.207553 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82765721711e484d8bc87340177657cc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.208361 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82765721711e484d8bc87340177657cc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.221210 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82765721711e484d8bc87340177657cc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.228257 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82765721711e484d8bc87340177657cc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.228933 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82765721711e484d8bc87340177657cc'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.234240 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:32.243486 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1448/2916] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:32 2022] DELETE /identity/v3/users/82765721711e484d8bc87340177657cc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:32.423277 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:32.435570 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b95c-374f-4783-a778-bc6ac48276d7 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:32.497701 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:32.498593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:32.498883 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:32.499154 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] PATH_INFO: `/v3/projects/a262c7be2bc040eea71787e95b11c92f` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:32.500355 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a262c7be2bc040eea71787e95b11c92f)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:32.502321 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:32.515051 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=100 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:32.553862 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a262c7be2bc040eea71787e95b11c92f', 'request_id': 'req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.559267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a262c7be2bc040eea71787e95b11c92f', 'request_id': 'req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.567299 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a262c7be2bc040eea71787e95b11c92f', 'request_id': 'req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.580267 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] The token cache is being invalidate because project a262c7be2bc040eea71787e95b11c92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:32.580551 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.581225 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.582908 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1469/2917] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:32 2022] DELETE /identity/v3/projects/a262c7be2bc040eea71787e95b11c92f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:32.587116 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:32.588526 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:32.603497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2da5f-7e07-4398-914a-b0b38b40af04 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:32.661357 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:32.662245 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:32.662550 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:32.663281 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] PATH_INFO: `/v3/users/e85c4d8a2c8441f69886848937b770f6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:32.664137 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e85c4d8a2c8441f69886848937b770f6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:32.666059 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:32.715326 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e85c4d8a2c8441f69886848937b770f6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.715991 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e85c4d8a2c8441f69886848937b770f6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.730961 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e85c4d8a2c8441f69886848937b770f6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.742316 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e85c4d8a2c8441f69886848937b770f6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.743148 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-ea8e7583-e5ad-4907-9f7c-7ef6825d0e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e85c4d8a2c8441f69886848937b770f6'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:32.758283 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1449/2918] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:32:32 2022] DELETE /identity/v3/users/e85c4d8a2c8441f69886848937b770f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:32.955676 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:32.957338 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:32.976636 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550d2b-c0d5-4d17-8e4d-6ebd43e8333b admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:33.047573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11331454-4655-485e-9856-2c3921358446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:33.048445 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331454-4655-485e-9856-2c3921358446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:33.048734 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331454-4655-485e-9856-2c3921358446 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:33.049007 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11331454-4655-485e-9856-2c3921358446 admin admin] PATH_INFO: `/v3/projects/7144a4c42655475e890db2a23d0517ac` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:33.050211 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11331454-4655-485e-9856-2c3921358446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7144a4c42655475e890db2a23d0517ac)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:33.052217 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11331454-4655-485e-9856-2c3921358446 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:33.110962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7144a4c42655475e890db2a23d0517ac', 'request_id': 'req-11331454-4655-485e-9856-2c3921358446'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:33.110962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7144a4c42655475e890db2a23d0517ac', 'request_id': 'req-11331454-4655-485e-9856-2c3921358446'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:33.116213 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7144a4c42655475e890db2a23d0517ac', 'request_id': 'req-11331454-4655-485e-9856-2c3921358446'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:33.128261 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] The token cache is being invalidate because project 7144a4c42655475e890db2a23d0517ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91851) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:32:33.128552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:33.129192 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:32:33.130767 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1470/2919] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:32:32 2022] DELETE /identity/v3/projects/7144a4c42655475e890db2a23d0517ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:32:40.519707 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:40.520330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:40.520330 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:40.538512 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] MFA Rules not processed for user `40078727842f491bb566e205065a65ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:32:40.539745 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:40.595472 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1450/2920] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:32:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:32:40.931629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:40.933377 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:40.966249 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:41.004043 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11331454-4655-485e-9856-2c3921358446 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:41.008349 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:41.008999 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:41.009280 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:41.009553 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:41.011378 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:41.022069 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:41.023536 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:41.032335 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1471/2921] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:32:43.496127 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:32:43.497579 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.519124 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:43.570081 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fb400-8cac-407e-9e18-a3ec53c2e35f None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:32:43.575140 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:32:43.575825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:32:43.576104 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:32:43.576379 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:32:43.578369 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:32:43.590325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:32:43.591955 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:32:43.603411 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1451/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:32:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:32:51.028024 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=13 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:32:53.598637 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=23 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:00.625372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:00.626956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:00.710254 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7b6fa-0adf-401c-ac78-8c60be593fc0 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:00.716018 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:00.716662 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:00.716997 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:00.717219 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:00.719723 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:00.731978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:00.733563 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cffe7-cbad-4d3a-9f3d-2ff0fc0449e8 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:00.744762 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1472/2923] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:03.741997 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:33:03.751733 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575ce6c-c7b4-4ff0-86f3-0a20a633d5a0 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:33:03.755918 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:33:03.756557 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:33:03.756840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:33:03.757117 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:33:03.758975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:33:03.827421 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:33:03.829386 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:33:03.839540 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1452/2924] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:33:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:33:10.740231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=34 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:33:13.834932 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=30 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:35:37.490670 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1112-4321-42de-beaf-57eead187b26 None None] REQUEST_METHOD: `POST` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:35:37.491244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1112-4321-42de-beaf-57eead187b26 None None] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:35:37.491244 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1112-4321-42de-beaf-57eead187b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:35:37.512832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.auth.core [None req-930b1112-4321-42de-beaf-57eead187b26 None None] MFA Rules not processed for user `40078727842f491bb566e205065a65ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91851) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:35:37.514006 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-930b1112-4321-42de-beaf-57eead187b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:37.521000 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1473/2925] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:35:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 18:35:37.841675 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:35:37.843780 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:37.855041 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7afe0-bfc1-4ce9-964b-049375c76fce service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:35:37.859259 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:35:37.859895 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:35:37.860176 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:35:37.860447 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:35:37.862220 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:35:37.872443 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:35:37.873902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:35:37.881975 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1453/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:35:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:35:47.507906 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=3 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:35:47.878381 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:22.831875 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b1112-4321-42de-beaf-57eead187b26 None None] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:22.833910 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-930b1112-4321-42de-beaf-57eead187b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:22.956537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b1112-4321-42de-beaf-57eead187b26 None None] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:22.965210 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:22.966303 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:22.966824 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:22.967309 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:22.970359 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:22.992956 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:22.996936 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:23.017300 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1474/2927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:33.009084 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=32 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:36.574205 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:36.583705 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2076a6-69d2-4c89-9fbe-164c1fa1d698 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:36.587774 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:36.588430 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:36.588714 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:36.588990 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:36.590971 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:36.601716 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:36.603496 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6b9aa-f561-4887-a08a-0f3a38766d59 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:36.613982 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1454/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:37:46.123608 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:46.125231 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:46.193263 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac36dba-7328-4371-af63-48ffc1ca056a service placement] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:46.197318 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:46.197996 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:46.198286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:46.198593 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:46.200755 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:46.214226 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:46.215731 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:46.227605 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1475/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:46.609624 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=6 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:56.222027 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=26 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:37:59.531514 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] REQUEST_METHOD: `POST` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:59.532394 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:59.532563 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:59.552023 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.auth.core [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] MFA Rules not processed for user `40078727842f491bb566e205065a65ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91852) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 18:37:59.553289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:59.565111 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1455/2930] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jun 30 18:37:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 18:37:59.607823 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:37:59.630119 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6000-6836-4fd2-b535-8db396c75363 service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:37:59.634101 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd5dd586ede14a918301fd93b4c8f320', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:37:59.634801 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:37:59.635081 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:37:59.635353 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:37:59.637281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:37:59.652703 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:37:59.654179 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:37:59.668068 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1476/2931] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:37:59.993825 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:00.005438 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739db59-e71b-476b-8ac3-c902079f0968 None None] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:00.009558 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:00.010273 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:00.010664 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:00.011072 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:00.020549 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:00.022288 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:00.033347 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1456/2932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:37:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:38:01.634832 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:01.645554 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda607d-a9b7-440a-acc6-2425d9aadcac service cinder] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:01.649289 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:01.649957 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:01.650241 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:01.650576 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:01.652514 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:01.668787 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:01.671143 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:01.684266 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1477/2933] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:38:02.629945 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:02.644489 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebe1a8-7cf3-4485-a644-9489ef8376b4 service neutron] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:02.649812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:02.650497 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:02.650784 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:02.651060 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:02.660842 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:02.662812 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:02.675101 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1457/2934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:38:08.832665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:08.864803 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c6338-8f27-47ce-bab3-b3f783e71acb service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:08.875775 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:08.877072 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:08.877646 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:08.878200 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:08.882640 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:08.911745 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:08.915847 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:08.946604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1478/2935] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:38:12.669437 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=17 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:18.933297 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=23 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:20.240532 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:20.243064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:20.268094 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae8085-aa9e-4329-8a50-523498ef6232 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:20.274662 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:20.275362 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:20.275811 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:20.276286 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:20.288672 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:20.290554 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:20.307633 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1458/2936] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:38:23.162384 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:23.185665 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210f62-07a7-424b-a1b7-ae3efd3ed683 service nova] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:23.190484 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:23.191157 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:23.191450 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:23.191725 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:23.202547 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:23.204026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:23.215873 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1479/2937] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:38:25.048289 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:38:25.061560 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da9ed6-d323-4242-99a3-8174ac969ff0 service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:38:25.073417 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c064d58c15704009864fc43a1e630676', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:38:25.074133 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:38:25.074458 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:38:25.074730 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:38:25.077383 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:38:25.090815 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:38:25.092533 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:38:25.105407 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1459/2938] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:38:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 18:38:33.210406 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=10 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:38:35.099490 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=12 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:28.602372 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:28.615611 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55232-52d2-432c-8699-da6b6f38114b service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:28.620039 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30dea44d32f0405a9344f004640d44a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:28.620735 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:28.621122 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:28.621442 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:28.623518 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:28.743697 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:28.746487 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:28.774019 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1480/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:32.728105 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:32.743442 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b3178-5cf1-40bf-9700-0e6a0dcb129f service nova] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:32.751191 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:32.752064 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] REQUEST_METHOD: `GET` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:32.752495 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:32.752937 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:32.755277 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:32.768926 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:32.770581 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:32.785629 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1460/2940] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:40:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:40:36.931286 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:36.933552 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:36.948320 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec15d1c-b6dd-449f-810a-d1175442d082 service neutron] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:36.954625 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:36.955976 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:36.956403 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:36.956826 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] PATH_INFO: `/v3/users/40078727842f491bb566e205065a65ad` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:36.958784 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40078727842f491bb566e205065a65ad)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:36.961290 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:37.034315 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40078727842f491bb566e205065a65ad'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.046698 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40078727842f491bb566e205065a65ad'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.047629 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40078727842f491bb566e205065a65ad'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.073136 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40078727842f491bb566e205065a65ad'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.074264 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40078727842f491bb566e205065a65ad'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.099521 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1481/2941] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:40:36 2022] DELETE /identity/v3/users/40078727842f491bb566e205065a65ad => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:40:37.334736 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:37.350949 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf84528-c928-430d-8b36-a1622218cac0 service placement] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:37.454775 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:37.456325 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:37.456885 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:37.457397 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] PATH_INFO: `/v3/projects/ad915a3e597f4730a63420d59470f5b6` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:37.459840 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad915a3e597f4730a63420d59470f5b6)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:37.463894 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:37.527061 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad915a3e597f4730a63420d59470f5b6', 'request_id': 'req-02732912-893f-4736-b6ee-8c3d572fa372'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.536091 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad915a3e597f4730a63420d59470f5b6', 'request_id': 'req-02732912-893f-4736-b6ee-8c3d572fa372'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.546474 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad915a3e597f4730a63420d59470f5b6', 'request_id': 'req-02732912-893f-4736-b6ee-8c3d572fa372'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.564308 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] The token cache is being invalidate because project ad915a3e597f4730a63420d59470f5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:40:37.564741 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.565521 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.567293 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1461/2942] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:40:37 2022] DELETE /identity/v3/projects/ad915a3e597f4730a63420d59470f5b6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 18:40:37.571619 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:37.573071 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:37.587329 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7d01-91dc-4977-a832-7b3811b47473 admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:37.628962 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:37.659255 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:37.660214 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:37.660603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:37.660973 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] PATH_INFO: `/v3/users/a0b85ec7b53a4230aff58e4b376ce163` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:37.662520 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b85ec7b53a4230aff58e4b376ce163)` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:37.664678 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:37.733840 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b85ec7b53a4230aff58e4b376ce163'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.745199 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b85ec7b53a4230aff58e4b376ce163'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.745913 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b85ec7b53a4230aff58e4b376ce163'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.761026 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b85ec7b53a4230aff58e4b376ce163'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.761780 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.notifications [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b85ec7b53a4230aff58e4b376ce163'} {{(pid=91851) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:37.777537 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1482/2943] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jun 30 18:40:37 2022] DELETE /identity/v3/users/a0b85ec7b53a4230aff58e4b376ce163 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:40:37.987727 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Authenticating user token {{(pid=91852) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:40:37.989603 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.fernet_utils [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91852) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:40:38.003618 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02732912-893f-4736-b6ee-8c3d572fa372 admin admin] Validating token access rules against request {{(pid=91852) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:40:38.077566 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c7b0226fd854776bbee2a2d0d0ee93a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b7acc57e2754d7ab425d395278a2937', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91852) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:40:38.078567 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:40:38.078902 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] SCRIPT_NAME: `/identity` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:40:38.079305 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] PATH_INFO: `/v3/projects/98c62fb5effe42cf8234cb62c82e05fc` {{(pid=91852) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:40:38.080802 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c62fb5effe42cf8234cb62c82e05fc)` {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:40:38.083051 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] RBAC: Authorization granted {{(pid=91852) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:40:38.111645 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:38.147744 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c62fb5effe42cf8234cb62c82e05fc', 'request_id': 'req-95c4dff7-f92f-4452-8d5a-48e48c3c0963'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:38.155294 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c62fb5effe42cf8234cb62c82e05fc', 'request_id': 'req-95c4dff7-f92f-4452-8d5a-48e48c3c0963'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:38.162182 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c62fb5effe42cf8234cb62c82e05fc', 'request_id': 'req-95c4dff7-f92f-4452-8d5a-48e48c3c0963'} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:38.177665 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] The token cache is being invalidate because project 98c62fb5effe42cf8234cb62c82e05fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91852) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 18:40:38.177903 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:38.178606 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG keystone.notifications [None req-95c4dff7-f92f-4452-8d5a-48e48c3c0963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91852) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 18:40:38.180287 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: [pid: 91852|app: 0|req: 1462/2944] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jun 30 18:40:37 2022] DELETE /identity/v3/projects/98c62fb5effe42cf8234cb62c82e05fc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 18:40:47.773419 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:40:48.173577 dv-cinder-812685-15-iscsi devstack@keystone.service[91852]: DEBUG dbcounter [-] [91852] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91852) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 18:42:25.801573 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Authenticating user token {{(pid=91851) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 18:42:25.803490 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:25.896978 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d986b-509d-4451-9e87-da890fd0b8cb admin admin] Validating token access rules against request {{(pid=91851) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 18:42:25.971921 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f20dd9dac7a4c4bb649e484d53a4928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4089a869f84cb59bf14cbf816a8fd3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91851) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 18:42:25.972604 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] REQUEST_METHOD: `GET` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 18:42:25.972891 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] SCRIPT_NAME: `/identity` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 18:42:25.973171 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91851) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 18:42:25.975327 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.fernet_utils [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91851) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 18:42:25.990603 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 18:42:25.993015 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd841-d2e1-4168-86d1-58140189641c service placement] RBAC: Authorization granted {{(pid=91851) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 18:42:26.009281 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: [pid: 91851|app: 0|req: 1483/2945] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jun 30 18:42:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 18:42:36.001316 dv-cinder-812685-15-iscsi devstack@keystone.service[91851]: DEBUG dbcounter [-] [91851] Writing DB stats keystone:SELECT=58 {{(pid=91851) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}